2024-04-23T18:17:35.259 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T18:17:35.259 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/direct_io}} --name leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi --verbose --owner scheduled_leonidus@teuthology -- /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122/orig.config.yaml 2024-04-23T18:17:35.315 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T18:17:35.402 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/direct_io}} email: null first_in_suite: false job_id: '7670122' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 57e0caf76fdb7ab8b1358588f292d08519163844 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5182 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 57e0caf76fdb7ab8b1358588f292d08519163844 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6S46nPASCb2Fit9kG0YfsJLawk4q0z5h9znWFv3gHZAVMFfRVOxJJyyJHp4RAeCP6G7iTZHEHNfwud0F0hePQ= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMtw0gz1uMEwr3c3QWDqdWtIIpp8x0ZSt7yhItFkbTefZO31ooSvs06nPaCT4TzLVb1jDYgEGLoFGL85CVuGwn0= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHyquATf+b7Sfm+cFw9lCwR72Volt5lK/yyz3fD3NfgKNtkHiujr8R4TIay9ILARYLayN1cgCg4vNtZ1QTcEXto= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_17:44:29 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-23T18:17:35.402 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa; will attempt to use it 2024-04-23T18:17:35.403 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks 2024-04-23T18:17:35.403 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T18:17:35.404 INFO:teuthology.task.internal:Checking packages... 2024-04-23T18:17:35.427 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-23T18:17:35.427 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T18:17:35.427 INFO:teuthology.packaging:ref: None 2024-04-23T18:17:35.427 INFO:teuthology.packaging:tag: None 2024-04-23T18:17:35.427 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-23T18:17:35.427 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:17:35.427 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-23T18:17:35.579 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-23T18:17:35.581 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T18:17:35.590 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T18:17:35.590 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T18:17:35.603 INFO:teuthology.task.internal:Saving configuration 2024-04-23T18:17:35.623 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T18:17:35.634 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T18:17:35.659 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:11:53.930190', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6S46nPASCb2Fit9kG0YfsJLawk4q0z5h9znWFv3gHZAVMFfRVOxJJyyJHp4RAeCP6G7iTZHEHNfwud0F0hePQ='} 2024-04-23T18:17:35.683 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:11:53.931935', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMtw0gz1uMEwr3c3QWDqdWtIIpp8x0ZSt7yhItFkbTefZO31ooSvs06nPaCT4TzLVb1jDYgEGLoFGL85CVuGwn0='} 2024-04-23T18:17:35.706 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:11:53.933544', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHyquATf+b7Sfm+cFw9lCwR72Volt5lK/yyz3fD3NfgKNtkHiujr8R4TIay9ILARYLayN1cgCg4vNtZ1QTcEXto='} 2024-04-23T18:17:35.706 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T18:17:35.717 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-23T18:17:35.717 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-23T18:17:35.717 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-23T18:17:35.717 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T18:17:35.824 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb26a8500d0>, signals=[15]) 2024-04-23T18:17:35.825 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T18:17:35.834 INFO:teuthology.task.internal:Opening connections... 2024-04-23T18:17:35.834 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2024-04-23T18:17:35.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:17:35.910 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-04-23T18:17:35.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:17:35.985 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-04-23T18:17:35.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:17:36.060 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T18:17:36.067 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2024-04-23T18:17:36.085 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2024-04-23T18:17:36.085 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2024-04-23T18:17:36.141 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2024-04-23T18:17:36.141 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="9" 2024-04-23T18:17:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2024-04-23T18:17:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2024-04-23T18:17:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="9" 2024-04-23T18:17:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el9" 2024-04-23T18:17:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T18:17:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2024-04-23T18:17:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:LOGO="fedora-logo-icon" 2024-04-23T18:17:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T18:17:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2024-04-23T18:17:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T18:17:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T18:17:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T18:17:36.143 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2024-04-23T18:17:36.163 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-04-23T18:17:36.181 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-04-23T18:17:36.181 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-04-23T18:17:36.238 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-04-23T18:17:36.238 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-04-23T18:17:36.238 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-04-23T18:17:36.238 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-04-23T18:17:36.238 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-04-23T18:17:36.238 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-04-23T18:17:36.238 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T18:17:36.238 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-04-23T18:17:36.239 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-04-23T18:17:36.239 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T18:17:36.239 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-04-23T18:17:36.239 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T18:17:36.239 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T18:17:36.239 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T18:17:36.239 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-04-23T18:17:36.263 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-04-23T18:17:36.280 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-04-23T18:17:36.281 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-04-23T18:17:36.338 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T18:17:36.339 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T18:17:36.340 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-04-23T18:17:36.363 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T18:17:36.378 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T18:17:36.453 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T18:17:36.454 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2024-04-23T18:17:36.456 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-04-23T18:17:36.459 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-04-23T18:17:36.474 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T18:17:36.485 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T18:17:36.485 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2024-04-23T18:17:36.513 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-04-23T18:17:36.515 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-04-23T18:17:36.790 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T18:17:36.873 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T18:17:36.890 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-23T18:17:36.890 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-23T18:17:36.890 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-23T18:17:36.890 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T18:17:36.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:17:36.907 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-04-23T18:17:36.965 INFO:teuthology.orchestra.run.smithi132.stdout:5.14.0-437.el9.x86_64 2024-04-23T18:17:36.965 INFO:teuthology.task.kernel:Running kernel on smithi132: 5.14.0-437.el9.x86_64 2024-04-23T18:17:36.965 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum install -y kernel 2024-04-23T18:17:41.657 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.0 MB 00:03 2024-04-23T18:17:59.273 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-04-23T18:18:04.198 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-04-23T18:18:06.443 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-04-23T18:18:10.251 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 21 MB 00:03 2024-04-23T18:18:15.234 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-23T18:18:17.107 INFO:teuthology.orchestra.run.smithi132.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T18:18:17.107 INFO:teuthology.orchestra.run.smithi132.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T18:18:17.186 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-04-23T18:18:17.187 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-04-23T18:18:17.188 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-04-23T18:18:17.313 DEBUG:teuthology.orchestra.run.smithi132:> echo no | sudo yum reinstall kernel || true 2024-04-23T18:18:17.811 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:02 ago on Tue 23 Apr 2024 06:18:15 PM UTC. 2024-04-23T18:18:17.942 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-04-23T18:18:17.956 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-04-23T18:18:17.956 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-04-23T18:18:17.956 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-04-23T18:18:17.956 INFO:teuthology.orchestra.run.smithi132.stdout:Reinstalling: 2024-04-23T18:18:17.956 INFO:teuthology.orchestra.run.smithi132.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T18:18:17.956 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:18:17.956 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-04-23T18:18:17.956 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-04-23T18:18:17.956 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:18:17.957 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 255 k 2024-04-23T18:18:17.957 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 0 2024-04-23T18:18:17.957 INFO:teuthology.orchestra.run.smithi132.stderr:Operation aborted. 2024-04-23T18:18:18.019 INFO:teuthology.orchestra.run.smithi132.stdout:Is this ok [y/N]: 2024-04-23T18:18:18.019 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum reinstall -y kernel || true 2024-04-23T18:18:18.528 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 06:18:15 PM UTC. 2024-04-23T18:18:18.655 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-04-23T18:18:18.655 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-04-23T18:18:18.655 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-04-23T18:18:18.655 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-04-23T18:18:18.656 INFO:teuthology.orchestra.run.smithi132.stdout:Reinstalling: 2024-04-23T18:18:18.656 INFO:teuthology.orchestra.run.smithi132.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T18:18:18.656 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:18:18.656 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-04-23T18:18:18.656 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-04-23T18:18:18.656 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:18:18.656 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 255 k 2024-04-23T18:18:18.657 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 0 2024-04-23T18:18:18.657 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-04-23T18:18:19.452 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-5.14.0-437.el9.x86_64.rpm 352 kB/s | 255 kB 00:00 2024-04-23T18:18:19.453 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:18:19.453 INFO:teuthology.orchestra.run.smithi132.stdout:Total 320 kB/s | 255 kB 00:00 2024-04-23T18:18:19.469 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-04-23T18:18:19.655 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-04-23T18:18:19.656 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-04-23T18:18:19.661 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-04-23T18:18:19.661 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-04-23T18:18:19.807 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-04-23T18:18:19.870 INFO:teuthology.orchestra.run.smithi132.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T18:18:19.870 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T18:18:20.513 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T18:18:20.513 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T18:18:21.192 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T18:18:21.193 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:18:21.193 INFO:teuthology.orchestra.run.smithi132.stdout:Reinstalled: 2024-04-23T18:18:21.193 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T18:18:21.193 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:18:21.193 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-04-23T18:18:21.366 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T18:18:21.413 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T18:18:21.413 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T18:18:21.414 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T18:18:21.414 DEBUG:teuthology.parallel:result is None 2024-04-23T18:18:21.414 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T18:18:21.424 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T18:18:21.424 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T18:18:21.426 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T18:18:21.429 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T18:18:21.447 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T18:18:21.458 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T18:18:21.472 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T18:18:21.472 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T18:18:21.481 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T18:18:21.486 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T18:18:21.616 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T18:18:21.627 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T18:18:21.627 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T18:18:21.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:18:21.644 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T18:18:21.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:18:21.661 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T18:18:21.663 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T18:18:21.687 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T18:18:21.698 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:18:21.720 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:18:21.731 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:18:21.735 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:18:21.742 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:18:21.768 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:18:21.770 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T18:18:21.778 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T18:18:21.779 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T18:18:21.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T18:18:21.783 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T18:18:21.837 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T18:18:21.849 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T18:18:21.849 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T18:18:21.851 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T18:18:21.854 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T18:18:21.893 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:18:21.944 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:18:21.990 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:18:22.071 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:18:22.109 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:18:22.110 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T18:18:22.173 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:18:22.217 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:18:22.270 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:18:22.353 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:18:22.397 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T18:18:22.397 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T18:18:22.464 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:18:22.508 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:18:22.565 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:18:22.648 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:18:22.691 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-23T18:18:22.691 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T18:18:22.757 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2024-04-23T18:18:22.760 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-04-23T18:18:22.762 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-04-23T18:18:22.793 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:18:22.794 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:18:22.838 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:18:23.323 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T18:18:23.331 INFO:teuthology.task.internal:Starting timer... 2024-04-23T18:18:23.331 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T18:18:23.347 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T18:18:23.360 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-23T18:18:23.360 DEBUG:teuthology.orchestra.run.smithi132:> sudo service auditd rotate 2024-04-23T18:18:23.410 INFO:teuthology.orchestra.run.smithi132.stdout:Rotating logs: 2024-04-23T18:18:23.412 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-04-23T18:18:23.480 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-04-23T18:18:23.482 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-04-23T18:18:23.534 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-04-23T18:18:23.536 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T18:18:23.536 DEBUG:teuthology.orchestra.run.smithi132:> /usr/sbin/getenforce 2024-04-23T18:18:23.553 INFO:teuthology.orchestra.run.smithi132.stdout:Permissive 2024-04-23T18:18:23.553 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-04-23T18:18:23.584 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-04-23T18:18:23.584 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-04-23T18:18:23.602 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-04-23T18:18:23.603 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi132.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-04-23T18:18:23.603 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:18:23.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:18:23.627 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:18:23.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:18:23.654 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:18:23.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:18:23.681 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T18:18:23.681 DEBUG:teuthology.orchestra.run.smithi132:> sudo /usr/sbin/setenforce permissive 2024-04-23T18:18:23.713 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-04-23T18:18:23.739 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-04-23T18:18:23.764 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T18:18:23.774 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-23T18:18:23.780 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-23T18:18:23.900 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T18:18:23.913 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T18:18:23.914 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi132.front.sepia.ceph.com,smithi148.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T18:22:21.393 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi132.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-04-23T18:22:21.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-23T18:22:21.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:22:21.471 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-04-23T18:22:21.552 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-23T18:22:21.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-04-23T18:22:21.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:22:21.627 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-04-23T18:22:21.705 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-04-23T18:22:21.705 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-23T18:22:21.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:22:21.780 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-04-23T18:22:21.865 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-23T18:22:21.866 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T18:22:21.877 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T18:22:21.877 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T18:22:21.877 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:22:21.879 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T18:22:21.880 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:22:21.882 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T18:22:21.882 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:22:21.905 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T18:22:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T18:22:21.918 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T18:22:21.921 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T18:22:21.933 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T18:22:21.940 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: ntpd: command not found 2024-04-23T18:22:21.943 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-04-23T18:22:21.946 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T18:22:21.950 INFO:teuthology.orchestra.run.smithi132.stdout:506 Cannot talk to daemon 2024-04-23T18:22:21.953 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-04-23T18:22:21.963 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T18:22:21.966 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T18:22:21.969 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-04-23T18:22:21.976 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T18:22:21.979 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T18:22:21.980 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-04-23T18:22:21.992 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T18:22:22.006 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T18:22:22.030 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:22:22.032 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:22:22.034 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:22:22.034 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-04-23T18:22:22.034 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:22:22.034 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:22:22.034 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:22:22.034 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:22:22.036 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:22:22.036 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-04-23T18:22:22.037 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:22:22.037 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:22:22.037 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:22:22.037 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:22:22.068 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:22:22.073 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:22:22.073 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-04-23T18:22:22.073 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:22:22.073 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:22:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:22:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:22:22.074 INFO:teuthology.run_tasks:Running task install... 2024-04-23T18:22:22.084 DEBUG:teuthology.task.install:project ceph 2024-04-23T18:22:22.084 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-23T18:22:22.084 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-23T18:22:22.084 INFO:teuthology.task.install:Using flavor: default 2024-04-23T18:22:22.090 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T18:22:22.090 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-23T18:22:22.090 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T18:22:22.091 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:22:22.093 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T18:22:22.093 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:22:22.094 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T18:22:22.094 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:22:22.247 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T18:22:22.247 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T18:22:22.305 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T18:22:22.306 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T18:22:22.317 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T18:22:22.318 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T18:22:22.421 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T18:22:22.421 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:22:22.421 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T18:22:22.427 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T18:22:22.428 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T18:22:22.428 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T18:22:22.445 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T18:22:22.445 DEBUG:teuthology.orchestra.run.smithi132:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T18:22:22.453 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T18:22:22.453 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T18:22:22.492 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T18:22:22.492 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-23T18:22:22.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T18:22:22.516 DEBUG:teuthology.orchestra.run.smithi132:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T18:22:22.517 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T18:22:22.518 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T18:22:22.519 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T18:22:22.582 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T18:22:22.589 DEBUG:teuthology.orchestra.run.smithi132:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T18:22:22.592 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T18:22:22.652 INFO:teuthology.orchestra.run.smithi132.stdout:check_obsoletes = 1 2024-04-23T18:22:22.654 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum clean all 2024-04-23T18:22:22.655 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T18:22:22.655 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-04-23T18:22:22.656 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-04-23T18:22:22.718 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-04-23T18:22:22.719 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-04-23T18:22:22.879 INFO:teuthology.orchestra.run.smithi132.stdout:48 files removed 2024-04-23T18:22:22.882 INFO:teuthology.orchestra.run.smithi148.stdout:48 files removed 2024-04-23T18:22:22.906 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T18:22:22.909 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T18:22:22.946 INFO:teuthology.orchestra.run.smithi157.stdout:48 files removed 2024-04-23T18:22:22.972 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T18:22:23.478 INFO:teuthology.orchestra.run.smithi132.stdout:ceph packages for x86_64 267 kB/s | 83 kB 00:00 2024-04-23T18:22:23.630 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 209 kB/s | 83 kB 00:00 2024-04-23T18:22:23.641 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 177 kB/s | 83 kB 00:00 2024-04-23T18:22:23.837 INFO:teuthology.orchestra.run.smithi132.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-23T18:22:23.988 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-23T18:22:24.075 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-23T18:22:24.212 INFO:teuthology.orchestra.run.smithi132.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-23T18:22:24.356 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-23T18:22:24.450 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-23T18:22:25.389 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 8.1 MB/s | 8.0 MB 00:00 2024-04-23T18:22:26.293 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.0 MB 00:01 2024-04-23T18:22:27.132 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-23T18:22:28.310 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-23T18:22:32.024 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.2 MB 00:01 2024-04-23T18:22:32.411 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-23T18:22:33.253 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 915 kB/s | 8.0 MB 00:09 2024-04-23T18:22:33.868 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-23T18:22:34.513 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 16 kB 00:00 2024-04-23T18:22:35.463 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-23T18:22:36.082 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-23T18:22:40.443 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-23T18:22:41.188 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-23T18:22:42.528 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:22:42.528 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:22:42.539 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T18:22:42.539 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T18:22:42.540 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T18:22:42.541 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T18:22:42.542 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T18:22:42.545 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T18:22:42.545 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T18:22:42.605 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-23T18:22:42.615 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-04-23T18:22:42.615 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T18:22:42.616 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T18:22:42.617 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T18:22:42.618 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T18:22:42.618 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T18:22:42.618 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T18:22:42.618 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T18:22:42.618 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T18:22:42.618 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T18:22:42.618 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T18:22:42.618 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:22:42.618 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T18:22:42.618 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T18:22:42.618 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T18:22:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T18:22:42.620 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T18:22:42.620 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T18:22:42.620 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T18:22:42.620 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T18:22:42.620 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T18:22:42.620 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T18:22:42.620 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T18:22:42.620 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T18:22:42.620 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T18:22:42.621 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T18:22:42.621 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T18:22:42.621 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T18:22:42.621 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T18:22:42.621 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T18:22:42.621 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T18:22:42.621 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T18:22:42.621 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T18:22:42.621 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T18:22:42.621 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T18:22:42.621 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:22:42.621 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T18:22:42.622 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T18:22:42.623 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T18:22:42.624 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T18:22:42.625 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T18:22:42.626 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T18:22:42.626 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T18:22:42.626 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T18:22:42.626 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T18:22:42.626 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T18:22:42.626 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T18:22:42.626 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T18:22:42.626 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T18:22:42.626 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-23T18:22:42.627 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-04-23T18:22:42.628 INFO:teuthology.orchestra.run.smithi157.stdout:Install 148 Packages 2024-04-23T18:22:42.628 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 4 Packages 2024-04-23T18:22:42.628 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:22:42.628 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 222 M 2024-04-23T18:22:42.628 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-23T18:22:43.289 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:22:43.290 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:22:43.300 INFO:teuthology.orchestra.run.smithi148.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T18:22:43.301 INFO:teuthology.orchestra.run.smithi148.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T18:22:43.302 INFO:teuthology.orchestra.run.smithi148.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T18:22:43.302 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T18:22:43.304 INFO:teuthology.orchestra.run.smithi148.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T18:22:43.306 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T18:22:43.307 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T18:22:43.367 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T18:22:43.381 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T18:22:43.382 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T18:22:43.383 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T18:22:43.384 INFO:teuthology.orchestra.run.smithi148.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T18:22:43.384 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T18:22:43.384 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T18:22:43.384 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T18:22:43.384 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T18:22:43.384 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T18:22:43.384 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T18:22:43.384 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T18:22:43.384 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T18:22:43.384 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T18:22:43.385 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T18:22:43.386 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T18:22:43.387 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T18:22:43.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T18:22:43.389 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T18:22:43.390 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T18:22:43.390 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T18:22:43.390 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T18:22:43.390 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T18:22:43.390 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T18:22:43.390 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T18:22:43.390 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T18:22:43.390 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T18:22:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T18:22:43.392 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T18:22:43.393 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-04-23T18:22:43.394 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T18:22:43.394 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:22:43.394 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T18:22:43.394 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-23T18:22:43.394 INFO:teuthology.orchestra.run.smithi148.stdout:Install 148 Packages 2024-04-23T18:22:43.394 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 4 Packages 2024-04-23T18:22:43.394 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:22:43.394 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 222 M 2024-04-23T18:22:43.395 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T18:22:43.536 INFO:teuthology.orchestra.run.smithi157.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-23T18:22:43.911 INFO:teuthology.orchestra.run.smithi157.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.3 MB/s | 890 kB 00:00 2024-04-23T18:22:44.012 INFO:teuthology.orchestra.run.smithi157.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.4 MB/s | 145 kB 00:00 2024-04-23T18:22:44.095 INFO:teuthology.orchestra.run.smithi157.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 6.6 MB/s | 5.4 MB 00:00 2024-04-23T18:22:44.354 INFO:teuthology.orchestra.run.smithi157.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 5.8 MB/s | 1.5 MB 00:00 2024-04-23T18:22:44.381 INFO:teuthology.orchestra.run.smithi148.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-23T18:22:44.471 INFO:teuthology.orchestra.run.smithi157.stdout:(6/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 5.1 MB/s | 2.4 MB 00:00 2024-04-23T18:22:45.281 INFO:teuthology.orchestra.run.smithi148.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 989 kB/s | 890 kB 00:00 2024-04-23T18:22:45.398 INFO:teuthology.orchestra.run.smithi148.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.2 MB/s | 145 kB 00:00 2024-04-23T18:22:45.404 INFO:teuthology.orchestra.run.smithi157.stdout:(7/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 10 MB/s | 21 MB 00:02 2024-04-23T18:22:45.471 INFO:teuthology.orchestra.run.smithi157.stdout:(8/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 4.2 MB/s | 4.7 MB 00:01 2024-04-23T18:22:45.596 INFO:teuthology.orchestra.run.smithi157.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 201 kB/s | 25 kB 00:00 2024-04-23T18:22:46.597 INFO:teuthology.orchestra.run.smithi157.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 8.9 MB/s | 11 MB 00:01 2024-04-23T18:22:46.755 INFO:teuthology.orchestra.run.smithi157.stdout:(11/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 7.4 MB/s | 17 MB 00:02 2024-04-23T18:22:46.780 INFO:teuthology.orchestra.run.smithi157.stdout:(12/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.2 MB/s | 225 kB 00:00 2024-04-23T18:22:46.782 INFO:teuthology.orchestra.run.smithi148.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.9 MB/s | 5.4 MB 00:02 2024-04-23T18:22:46.849 INFO:teuthology.orchestra.run.smithi157.stdout:(13/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 290 kB/s | 27 kB 00:00 2024-04-23T18:22:46.872 INFO:teuthology.orchestra.run.smithi157.stdout:(14/152): libcephfs-devel-19.0.0-3156.gd69f13d8 344 kB/s | 32 kB 00:00 2024-04-23T18:22:46.948 INFO:teuthology.orchestra.run.smithi157.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.1 MB/s | 163 kB 00:00 2024-04-23T18:22:46.990 INFO:teuthology.orchestra.run.smithi157.stdout:(16/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 5.2 MB/s | 747 kB 00:00 2024-04-23T18:22:47.015 INFO:teuthology.orchestra.run.smithi157.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 1.8 MB/s | 126 kB 00:00 2024-04-23T18:22:47.074 INFO:teuthology.orchestra.run.smithi157.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 5.9 MB/s | 500 kB 00:00 2024-04-23T18:22:47.174 INFO:teuthology.orchestra.run.smithi157.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 451 kB/s | 45 kB 00:00 2024-04-23T18:22:47.182 INFO:teuthology.orchestra.run.smithi148.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-23T18:22:47.241 INFO:teuthology.orchestra.run.smithi148.stdout:(6/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.3 MB/s | 2.4 MB 00:01 2024-04-23T18:22:47.258 INFO:teuthology.orchestra.run.smithi157.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.6 MB/s | 135 kB 00:00 2024-04-23T18:22:47.342 INFO:teuthology.orchestra.run.smithi157.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.9 MB/s | 161 kB 00:00 2024-04-23T18:22:47.426 INFO:teuthology.orchestra.run.smithi157.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.8 MB/s | 320 kB 00:00 2024-04-23T18:22:47.509 INFO:teuthology.orchestra.run.smithi157.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.5 MB/s | 298 kB 00:00 2024-04-23T18:22:47.593 INFO:teuthology.orchestra.run.smithi157.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.2 MB/s | 99 kB 00:00 2024-04-23T18:22:47.668 INFO:teuthology.orchestra.run.smithi157.stdout:(25/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 8.2 MB/s | 5.4 MB 00:00 2024-04-23T18:22:47.710 INFO:teuthology.orchestra.run.smithi157.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 730 kB/s | 85 kB 00:00 2024-04-23T18:22:47.785 INFO:teuthology.orchestra.run.smithi157.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.2 MB/s | 170 kB 00:00 2024-04-23T18:22:47.852 INFO:teuthology.orchestra.run.smithi157.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 514 kB/s | 34 kB 00:00 2024-04-23T18:22:47.936 INFO:teuthology.orchestra.run.smithi157.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.7 MB/s | 142 kB 00:00 2024-04-23T18:22:48.078 INFO:teuthology.orchestra.run.smithi157.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 7.5 MB/s | 3.1 MB 00:00 2024-04-23T18:22:48.345 INFO:teuthology.orchestra.run.smithi157.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 8.7 MB/s | 3.5 MB 00:00 2024-04-23T18:22:48.453 INFO:teuthology.orchestra.run.smithi157.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.3 MB/s | 249 kB 00:00 2024-04-23T18:22:48.520 INFO:teuthology.orchestra.run.smithi157.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 743 kB/s | 49 kB 00:00 2024-04-23T18:22:48.595 INFO:teuthology.orchestra.run.smithi157.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 219 kB/s | 16 kB 00:00 2024-04-23T18:22:48.666 INFO:teuthology.orchestra.run.smithi148.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.2 MB/s | 4.7 MB 00:01 2024-04-23T18:22:48.671 INFO:teuthology.orchestra.run.smithi157.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.6 MB/s | 273 kB 00:00 2024-04-23T18:22:48.780 INFO:teuthology.orchestra.run.smithi157.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 6.9 MB/s | 762 kB 00:00 2024-04-23T18:22:48.855 INFO:teuthology.orchestra.run.smithi157.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 243 kB/s | 18 kB 00:00 2024-04-23T18:22:48.883 INFO:teuthology.orchestra.run.smithi148.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 4.3 MB/s | 21 MB 00:04 2024-04-23T18:22:48.933 INFO:teuthology.orchestra.run.smithi148.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 508 kB/s | 25 kB 00:00 2024-04-23T18:22:48.947 INFO:teuthology.orchestra.run.smithi157.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 8.5 MB/s | 7.4 MB 00:00 2024-04-23T18:22:48.986 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-04-23T18:22:49.030 INFO:teuthology.orchestra.run.smithi157.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 459 kB/s | 80 kB 00:00 2024-04-23T18:22:49.097 INFO:teuthology.orchestra.run.smithi157.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-23T18:22:49.148 INFO:teuthology.orchestra.run.smithi157.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-23T18:22:49.175 INFO:teuthology.orchestra.run.smithi157.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-04-23T18:22:49.225 INFO:teuthology.orchestra.run.smithi157.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-23T18:22:49.259 INFO:teuthology.orchestra.run.smithi157.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-23T18:22:49.284 INFO:teuthology.orchestra.run.smithi157.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-23T18:22:49.309 INFO:teuthology.orchestra.run.smithi157.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-23T18:22:49.335 INFO:teuthology.orchestra.run.smithi157.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-23T18:22:49.477 INFO:teuthology.orchestra.run.smithi157.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-04-23T18:22:49.511 INFO:teuthology.orchestra.run.smithi157.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 5.3 MB/s | 943 kB 00:00 2024-04-23T18:22:49.535 INFO:teuthology.orchestra.run.smithi157.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-23T18:22:49.561 INFO:teuthology.orchestra.run.smithi157.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-23T18:22:49.586 INFO:teuthology.orchestra.run.smithi157.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-23T18:22:49.611 INFO:teuthology.orchestra.run.smithi157.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-04-23T18:22:49.653 INFO:teuthology.orchestra.run.smithi157.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-04-23T18:22:49.713 INFO:teuthology.orchestra.run.smithi157.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-23T18:22:49.738 INFO:teuthology.orchestra.run.smithi157.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-04-23T18:22:49.762 INFO:teuthology.orchestra.run.smithi157.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-23T18:22:49.788 INFO:teuthology.orchestra.run.smithi157.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-23T18:22:49.822 INFO:teuthology.orchestra.run.smithi157.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-23T18:22:49.847 INFO:teuthology.orchestra.run.smithi157.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-04-23T18:22:49.872 INFO:teuthology.orchestra.run.smithi157.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 508 kB/s | 25 kB 00:00 2024-04-23T18:22:49.897 INFO:teuthology.orchestra.run.smithi157.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-23T18:22:49.923 INFO:teuthology.orchestra.run.smithi157.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-23T18:22:49.939 INFO:teuthology.orchestra.run.smithi157.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-04-23T18:22:49.956 INFO:teuthology.orchestra.run.smithi157.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 35 kB 00:00 2024-04-23T18:22:50.065 INFO:teuthology.orchestra.run.smithi157.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-23T18:22:50.124 INFO:teuthology.orchestra.run.smithi157.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-04-23T18:22:50.166 INFO:teuthology.orchestra.run.smithi157.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-23T18:22:50.199 INFO:teuthology.orchestra.run.smithi157.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-23T18:22:50.224 INFO:teuthology.orchestra.run.smithi157.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-23T18:22:50.275 INFO:teuthology.orchestra.run.smithi157.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-04-23T18:22:50.350 INFO:teuthology.orchestra.run.smithi148.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 6.3 MB/s | 11 MB 00:01 2024-04-23T18:22:50.350 INFO:teuthology.orchestra.run.smithi157.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 38 MB/s | 5.8 MB 00:00 2024-04-23T18:22:50.376 INFO:teuthology.orchestra.run.smithi157.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 875 kB/s | 87 kB 00:00 2024-04-23T18:22:50.416 INFO:teuthology.orchestra.run.smithi148.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 3.3 MB/s | 225 kB 00:00 2024-04-23T18:22:50.418 INFO:teuthology.orchestra.run.smithi157.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-23T18:22:50.467 INFO:teuthology.orchestra.run.smithi148.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 542 kB/s | 27 kB 00:00 2024-04-23T18:22:50.493 INFO:teuthology.orchestra.run.smithi157.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 42 MB/s | 6.0 MB 00:00 2024-04-23T18:22:50.517 INFO:teuthology.orchestra.run.smithi148.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 634 kB/s | 32 kB 00:00 2024-04-23T18:22:50.518 INFO:teuthology.orchestra.run.smithi157.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-23T18:22:50.544 INFO:teuthology.orchestra.run.smithi157.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-04-23T18:22:50.569 INFO:teuthology.orchestra.run.smithi157.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-23T18:22:50.594 INFO:teuthology.orchestra.run.smithi157.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-23T18:22:50.619 INFO:teuthology.orchestra.run.smithi157.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-04-23T18:22:50.659 INFO:teuthology.orchestra.run.smithi148.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 5.2 MB/s | 747 kB 00:00 2024-04-23T18:22:50.661 INFO:teuthology.orchestra.run.smithi157.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-04-23T18:22:50.717 INFO:teuthology.orchestra.run.smithi148.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.7 MB/s | 163 kB 00:00 2024-04-23T18:22:50.728 INFO:teuthology.orchestra.run.smithi157.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-04-23T18:22:50.753 INFO:teuthology.orchestra.run.smithi157.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-23T18:22:50.768 INFO:teuthology.orchestra.run.smithi148.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 2.5 MB/s | 126 kB 00:00 2024-04-23T18:22:50.779 INFO:teuthology.orchestra.run.smithi157.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-23T18:22:50.804 INFO:teuthology.orchestra.run.smithi157.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 838 kB/s | 42 kB 00:00 2024-04-23T18:22:50.830 INFO:teuthology.orchestra.run.smithi157.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-23T18:22:50.855 INFO:teuthology.orchestra.run.smithi157.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-23T18:22:50.885 INFO:teuthology.orchestra.run.smithi148.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 4.2 MB/s | 500 kB 00:00 2024-04-23T18:22:50.888 INFO:teuthology.orchestra.run.smithi157.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-04-23T18:22:50.914 INFO:teuthology.orchestra.run.smithi157.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 868 kB/s | 51 kB 00:00 2024-04-23T18:22:50.941 INFO:teuthology.orchestra.run.smithi157.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-23T18:22:50.981 INFO:teuthology.orchestra.run.smithi157.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-23T18:22:51.023 INFO:teuthology.orchestra.run.smithi157.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-04-23T18:22:51.065 INFO:teuthology.orchestra.run.smithi157.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-23T18:22:51.107 INFO:teuthology.orchestra.run.smithi157.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-04-23T18:22:51.266 INFO:teuthology.orchestra.run.smithi157.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-04-23T18:22:51.291 INFO:teuthology.orchestra.run.smithi157.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-04-23T18:22:51.316 INFO:teuthology.orchestra.run.smithi157.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-23T18:22:51.342 INFO:teuthology.orchestra.run.smithi157.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-23T18:22:51.358 INFO:teuthology.orchestra.run.smithi157.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-23T18:22:51.376 INFO:teuthology.orchestra.run.smithi157.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 64 kB 00:00 2024-04-23T18:22:51.393 INFO:teuthology.orchestra.run.smithi157.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-23T18:22:51.434 INFO:teuthology.orchestra.run.smithi157.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 382 kB/s | 22 kB 00:00 2024-04-23T18:22:51.568 INFO:teuthology.orchestra.run.smithi157.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-04-23T18:22:51.677 INFO:teuthology.orchestra.run.smithi148.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 6.8 MB/s | 5.4 MB 00:00 2024-04-23T18:22:51.719 INFO:teuthology.orchestra.run.smithi148.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 1.1 MB/s | 45 kB 00:00 2024-04-23T18:22:51.769 INFO:teuthology.orchestra.run.smithi148.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 2.6 MB/s | 135 kB 00:00 2024-04-23T18:22:51.828 INFO:teuthology.orchestra.run.smithi148.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.7 MB/s | 161 kB 00:00 2024-04-23T18:22:51.912 INFO:teuthology.orchestra.run.smithi148.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.8 MB/s | 320 kB 00:00 2024-04-23T18:22:51.995 INFO:teuthology.orchestra.run.smithi148.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.5 MB/s | 298 kB 00:00 2024-04-23T18:22:52.054 INFO:teuthology.orchestra.run.smithi148.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 99 kB 00:00 2024-04-23T18:22:52.112 INFO:teuthology.orchestra.run.smithi148.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 85 kB 00:00 2024-04-23T18:22:52.360 INFO:teuthology.orchestra.run.smithi157.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 390 kB/s | 309 kB 00:00 2024-04-23T18:22:52.485 INFO:teuthology.orchestra.run.smithi157.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 522 kB/s | 548 kB 00:01 2024-04-23T18:22:52.511 INFO:teuthology.orchestra.run.smithi157.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 130 kB/s | 19 kB 00:00 2024-04-23T18:22:52.605 INFO:teuthology.orchestra.run.smithi148.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 6.3 MB/s | 3.1 MB 00:00 2024-04-23T18:22:52.646 INFO:teuthology.orchestra.run.smithi148.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 4.0 MB/s | 170 kB 00:00 2024-04-23T18:22:52.686 INFO:teuthology.orchestra.run.smithi157.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 148 kB/s | 26 kB 00:00 2024-04-23T18:22:52.705 INFO:teuthology.orchestra.run.smithi148.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 587 kB/s | 34 kB 00:00 2024-04-23T18:22:52.755 INFO:teuthology.orchestra.run.smithi148.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.8 MB/s | 142 kB 00:00 2024-04-23T18:22:52.836 INFO:teuthology.orchestra.run.smithi157.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 324 kB/s | 49 kB 00:00 2024-04-23T18:22:53.011 INFO:teuthology.orchestra.run.smithi157.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 385 kB/s | 67 kB 00:00 2024-04-23T18:22:53.261 INFO:teuthology.orchestra.run.smithi157.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 604 kB/s | 151 kB 00:00 2024-04-23T18:22:53.537 INFO:teuthology.orchestra.run.smithi157.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-04-23T18:22:53.605 INFO:teuthology.orchestra.run.smithi157.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.4 MB/s | 838 kB 00:00 2024-04-23T18:22:53.631 INFO:teuthology.orchestra.run.smithi148.stdout:(30/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 10 MB/s | 49 MB 00:04 2024-04-23T18:22:53.689 INFO:teuthology.orchestra.run.smithi148.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 3.8 MB/s | 3.5 MB 00:00 2024-04-23T18:22:53.737 INFO:teuthology.orchestra.run.smithi157.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 326 kB/s | 43 kB 00:00 2024-04-23T18:22:53.762 INFO:teuthology.orchestra.run.smithi157.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-04-23T18:22:53.848 INFO:teuthology.orchestra.run.smithi148.stdout:(32/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 2.5 MB/s | 17 MB 00:06 2024-04-23T18:22:53.873 INFO:teuthology.orchestra.run.smithi148.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.3 MB/s | 249 kB 00:00 2024-04-23T18:22:53.923 INFO:teuthology.orchestra.run.smithi148.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 329 kB/s | 16 kB 00:00 2024-04-23T18:22:53.965 INFO:teuthology.orchestra.run.smithi148.stdout:(35/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 424 kB/s | 49 kB 00:00 2024-04-23T18:22:54.011 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-04-23T18:22:54.015 INFO:teuthology.orchestra.run.smithi148.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 273 kB 00:00 2024-04-23T18:22:54.066 INFO:teuthology.orchestra.run.smithi148.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 365 kB/s | 18 kB 00:00 2024-04-23T18:22:54.174 INFO:teuthology.orchestra.run.smithi148.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-23T18:22:54.208 INFO:teuthology.orchestra.run.smithi148.stdout:(39/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 3.1 MB/s | 762 kB 00:00 2024-04-23T18:22:54.229 INFO:teuthology.orchestra.run.smithi157.stdout:(115/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 5.7 MB/s | 49 MB 00:08 2024-04-23T18:22:54.255 INFO:teuthology.orchestra.run.smithi157.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 30 kB/s | 15 kB 00:00 2024-04-23T18:22:54.275 INFO:teuthology.orchestra.run.smithi148.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.8 MB/s | 803 kB 00:00 2024-04-23T18:22:54.280 INFO:teuthology.orchestra.run.smithi157.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 59 kB/s | 32 kB 00:00 2024-04-23T18:22:54.347 INFO:teuthology.orchestra.run.smithi157.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-23T18:22:54.359 INFO:teuthology.orchestra.run.smithi148.stdout:(41/152): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-23T18:22:54.384 INFO:teuthology.orchestra.run.smithi148.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-23T18:22:54.417 INFO:teuthology.orchestra.run.smithi148.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-23T18:22:54.422 INFO:teuthology.orchestra.run.smithi157.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-04-23T18:22:54.443 INFO:teuthology.orchestra.run.smithi148.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-23T18:22:54.468 INFO:teuthology.orchestra.run.smithi148.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-23T18:22:54.480 INFO:teuthology.orchestra.run.smithi157.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 264 kB/s | 35 kB 00:00 2024-04-23T18:22:54.493 INFO:teuthology.orchestra.run.smithi148.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-23T18:22:54.527 INFO:teuthology.orchestra.run.smithi148.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-23T18:22:54.556 INFO:teuthology.orchestra.run.smithi157.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-04-23T18:22:54.594 INFO:teuthology.orchestra.run.smithi148.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-04-23T18:22:54.682 INFO:teuthology.orchestra.run.smithi157.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-23T18:22:54.798 INFO:teuthology.orchestra.run.smithi157.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-23T18:22:54.849 INFO:teuthology.orchestra.run.smithi157.stdout:(124/152): python3-cherrypy-18.6.1-2.el9.noarch 604 kB/s | 358 kB 00:00 2024-04-23T18:22:54.882 INFO:teuthology.orchestra.run.smithi157.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.1 MB/s | 2.0 MB 00:00 2024-04-23T18:22:54.908 INFO:teuthology.orchestra.run.smithi157.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-04-23T18:22:54.983 INFO:teuthology.orchestra.run.smithi157.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 173 kB/s | 23 kB 00:00 2024-04-23T18:22:55.054 INFO:teuthology.orchestra.run.smithi157.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 113 kB/s | 19 kB 00:00 2024-04-23T18:22:55.108 INFO:teuthology.orchestra.run.smithi157.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 98 kB/s | 20 kB 00:00 2024-04-23T18:22:55.225 INFO:teuthology.orchestra.run.smithi157.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 397 kB/s | 46 kB 00:00 2024-04-23T18:22:55.250 INFO:teuthology.orchestra.run.smithi157.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 556 kB/s | 79 kB 00:00 2024-04-23T18:22:55.309 INFO:teuthology.orchestra.run.smithi157.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 3.1 MB/s | 1.0 MB 00:00 2024-04-23T18:22:55.336 INFO:teuthology.orchestra.run.smithi148.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 144 kB/s | 107 kB 00:00 2024-04-23T18:22:55.342 INFO:teuthology.orchestra.run.smithi157.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 493 kB/s | 58 kB 00:00 2024-04-23T18:22:55.410 INFO:teuthology.orchestra.run.smithi157.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-04-23T18:22:55.420 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-04-23T18:22:55.435 INFO:teuthology.orchestra.run.smithi157.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-04-23T18:22:55.535 INFO:teuthology.orchestra.run.smithi157.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 469 kB/s | 90 kB 00:00 2024-04-23T18:22:55.560 INFO:teuthology.orchestra.run.smithi157.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 187 kB/s | 28 kB 00:00 2024-04-23T18:22:55.585 INFO:teuthology.orchestra.run.smithi157.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 204 kB/s | 31 kB 00:00 2024-04-23T18:22:55.719 INFO:teuthology.orchestra.run.smithi157.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-04-23T18:22:55.744 INFO:teuthology.orchestra.run.smithi157.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 226 kB/s | 36 kB 00:00 2024-04-23T18:22:55.770 INFO:teuthology.orchestra.run.smithi157.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 322 kB/s | 59 kB 00:00 2024-04-23T18:22:55.853 INFO:teuthology.orchestra.run.smithi157.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 381 kB/s | 51 kB 00:00 2024-04-23T18:22:55.911 INFO:teuthology.orchestra.run.smithi148.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 37 kB/s | 21 kB 00:00 2024-04-23T18:22:55.937 INFO:teuthology.orchestra.run.smithi157.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 537 kB/s | 90 kB 00:00 2024-04-23T18:22:55.978 INFO:teuthology.orchestra.run.smithi148.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 464 kB/s | 689 kB 00:01 2024-04-23T18:22:56.021 INFO:teuthology.orchestra.run.smithi157.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 833 kB/s | 230 kB 00:00 2024-04-23T18:22:56.046 INFO:teuthology.orchestra.run.smithi157.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-23T18:22:56.080 INFO:teuthology.orchestra.run.smithi157.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-04-23T18:22:56.163 INFO:teuthology.orchestra.run.smithi157.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-04-23T18:22:56.386 INFO:teuthology.orchestra.run.smithi148.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 507 kB/s | 943 kB 00:01 2024-04-23T18:22:56.588 INFO:teuthology.orchestra.run.smithi157.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 2.9 MB/s | 1.6 MB 00:00 2024-04-23T18:22:56.638 INFO:teuthology.orchestra.run.smithi157.stdout:(149/152): librados2-19.0.0-3156.gd69f13d8.el7. 6.0 MB/s | 3.4 MB 00:00 2024-04-23T18:22:56.670 INFO:teuthology.orchestra.run.smithi148.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 47 kB/s | 33 kB 00:00 2024-04-23T18:22:56.839 INFO:teuthology.orchestra.run.smithi157.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 5.4 MB/s | 1.3 MB 00:00 2024-04-23T18:22:56.889 INFO:teuthology.orchestra.run.smithi157.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 4.3 MB/s | 3.1 MB 00:00 2024-04-23T18:22:56.956 INFO:teuthology.orchestra.run.smithi157.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.6 MB/s | 2.4 MB 00:00 2024-04-23T18:22:56.959 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:22:56.959 INFO:teuthology.orchestra.run.smithi157.stdout:Total 15 MB/s | 222 MB 00:14 2024-04-23T18:22:57.054 INFO:teuthology.orchestra.run.smithi148.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 275 kB/s | 313 kB 00:01 2024-04-23T18:22:57.229 INFO:teuthology.orchestra.run.smithi148.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 34 kB/s | 18 kB 00:00 2024-04-23T18:22:57.364 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-23T18:22:57.604 INFO:teuthology.orchestra.run.smithi148.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 20 kB/s | 11 kB 00:00 2024-04-23T18:22:58.063 INFO:teuthology.orchestra.run.smithi148.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 192 kB/s | 160 kB 00:00 2024-04-23T18:22:58.080 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-23T18:22:58.223 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-23T18:22:58.223 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-23T18:22:58.238 INFO:teuthology.orchestra.run.smithi148.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-04-23T18:22:59.622 INFO:teuthology.orchestra.run.smithi148.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 421 kB/s | 582 kB 00:01 2024-04-23T18:22:59.647 INFO:teuthology.orchestra.run.smithi148.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 418 kB/s | 662 kB 00:01 2024-04-23T18:22:59.998 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-23T18:22:59.999 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-23T18:23:00.205 INFO:teuthology.orchestra.run.smithi148.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 44 kB/s | 25 kB 00:00 2024-04-23T18:23:00.272 INFO:teuthology.orchestra.run.smithi148.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 76 kB/s | 47 kB 00:00 2024-04-23T18:23:00.972 INFO:teuthology.orchestra.run.smithi148.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 668 kB/s | 3.0 MB 00:04 2024-04-23T18:23:01.165 INFO:teuthology.orchestra.run.smithi148.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 211 kB/s | 188 kB 00:00 2024-04-23T18:23:01.198 INFO:teuthology.orchestra.run.smithi148.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 295 kB/s | 292 kB 00:00 2024-04-23T18:23:01.582 INFO:teuthology.orchestra.run.smithi148.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 58 kB/s | 35 kB 00:00 2024-04-23T18:23:02.430 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-23T18:23:04.373 INFO:teuthology.orchestra.run.smithi148.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 583 kB/s | 1.6 MB 00:02 2024-04-23T18:23:04.539 INFO:teuthology.orchestra.run.smithi132.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:23:04.540 INFO:teuthology.orchestra.run.smithi132.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:23:04.550 INFO:teuthology.orchestra.run.smithi132.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T18:23:04.551 INFO:teuthology.orchestra.run.smithi132.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T18:23:04.552 INFO:teuthology.orchestra.run.smithi132.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T18:23:04.553 INFO:teuthology.orchestra.run.smithi132.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T18:23:04.554 INFO:teuthology.orchestra.run.smithi132.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T18:23:04.556 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T18:23:04.557 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T18:23:04.617 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-04-23T18:23:04.626 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================= 2024-04-23T18:23:04.626 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-04-23T18:23:04.626 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================= 2024-04-23T18:23:04.626 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-04-23T18:23:04.626 INFO:teuthology.orchestra.run.smithi132.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T18:23:04.626 INFO:teuthology.orchestra.run.smithi132.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T18:23:04.626 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:23:04.626 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T18:23:04.626 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T18:23:04.627 INFO:teuthology.orchestra.run.smithi132.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T18:23:04.628 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T18:23:04.629 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T18:23:04.630 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T18:23:04.631 INFO:teuthology.orchestra.run.smithi132.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T18:23:04.632 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T18:23:04.633 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T18:23:04.634 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T18:23:04.635 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T18:23:04.636 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout:Installing weak dependencies: 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================= 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout:Install 148 Packages 2024-04-23T18:23:04.637 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 4 Packages 2024-04-23T18:23:04.638 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:23:04.638 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 222 M 2024-04-23T18:23:04.638 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-04-23T18:23:05.207 INFO:teuthology.orchestra.run.smithi132.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 41 kB/s | 6.6 kB 00:00 2024-04-23T18:23:05.415 INFO:teuthology.orchestra.run.smithi132.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 4.2 MB/s | 890 kB 00:00 2024-04-23T18:23:05.465 INFO:teuthology.orchestra.run.smithi132.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.8 MB/s | 145 kB 00:00 2024-04-23T18:23:05.699 INFO:teuthology.orchestra.run.smithi132.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 10 MB/s | 2.4 MB 00:00 2024-04-23T18:23:05.857 INFO:teuthology.orchestra.run.smithi132.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 9.5 MB/s | 1.5 MB 00:00 2024-04-23T18:23:05.942 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-23T18:23:06.017 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T18:23:06.058 INFO:teuthology.orchestra.run.smithi132.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 5.3 MB/s | 5.4 MB 00:01 2024-04-23T18:23:06.098 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T18:23:06.123 INFO:teuthology.orchestra.run.smithi148.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 521 kB/s | 911 kB 00:01 2024-04-23T18:23:06.308 INFO:teuthology.orchestra.run.smithi132.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 10 MB/s | 4.7 MB 00:00 2024-04-23T18:23:06.320 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T18:23:06.361 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T18:23:06.386 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T18:23:06.430 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T18:23:06.783 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T18:23:06.903 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:23:06.957 INFO:teuthology.orchestra.run.smithi148.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 160 kB/s | 133 kB 00:00 2024-04-23T18:23:07.025 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:23:07.054 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:23:07.146 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:23:07.183 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:23:07.316 INFO:teuthology.orchestra.run.smithi132.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 11 MB/s | 11 MB 00:01 2024-04-23T18:23:07.335 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T18:23:07.380 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T18:23:07.392 INFO:teuthology.orchestra.run.smithi132.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 336 kB/s | 25 kB 00:00 2024-04-23T18:23:07.408 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:23:07.502 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:23:07.526 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:23:07.565 INFO:teuthology.orchestra.run.smithi148.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 711 kB/s | 4.4 MB 00:06 2024-04-23T18:23:07.588 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:23:07.610 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:23:07.632 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:23:07.824 INFO:teuthology.orchestra.run.smithi148.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 729 kB/s | 4.7 MB 00:06 2024-04-23T18:23:07.827 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:23:07.860 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:23:07.970 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T18:23:08.064 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T18:23:08.101 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T18:23:08.117 INFO:teuthology.orchestra.run.smithi132.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 6.9 MB/s | 21 MB 00:03 2024-04-23T18:23:08.136 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T18:23:08.210 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T18:23:08.252 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T18:23:08.292 INFO:teuthology.orchestra.run.smithi132.stdout:(11/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 7.5 MB/s | 17 MB 00:02 2024-04-23T18:23:08.296 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T18:23:08.317 INFO:teuthology.orchestra.run.smithi132.stdout:(12/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.1 MB/s | 225 kB 00:00 2024-04-23T18:23:08.331 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T18:23:08.351 INFO:teuthology.orchestra.run.smithi132.stdout:(13/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 463 kB/s | 27 kB 00:00 2024-04-23T18:23:08.355 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T18:23:08.378 INFO:teuthology.orchestra.run.smithi132.stdout:(14/152): libcephfs-devel-19.0.0-3156.gd69f13d8 524 kB/s | 32 kB 00:00 2024-04-23T18:23:08.427 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T18:23:08.462 INFO:teuthology.orchestra.run.smithi132.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.9 MB/s | 163 kB 00:00 2024-04-23T18:23:08.495 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T18:23:08.512 INFO:teuthology.orchestra.run.smithi132.stdout:(16/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 4.5 MB/s | 747 kB 00:00 2024-04-23T18:23:08.525 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T18:23:08.546 INFO:teuthology.orchestra.run.smithi132.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-23T18:23:08.568 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T18:23:08.605 INFO:teuthology.orchestra.run.smithi132.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 5.3 MB/s | 500 kB 00:00 2024-04-23T18:23:08.607 INFO:teuthology.orchestra.run.smithi148.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 116 kB/s | 87 kB 00:00 2024-04-23T18:23:08.645 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T18:23:08.679 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T18:23:08.680 INFO:teuthology.orchestra.run.smithi132.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 602 kB/s | 45 kB 00:00 2024-04-23T18:23:08.722 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T18:23:08.751 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T18:23:08.763 INFO:teuthology.orchestra.run.smithi132.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.6 MB/s | 135 kB 00:00 2024-04-23T18:23:08.822 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T18:23:08.866 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T18:23:08.880 INFO:teuthology.orchestra.run.smithi132.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.4 MB/s | 161 kB 00:00 2024-04-23T18:23:08.941 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T18:23:08.981 INFO:teuthology.orchestra.run.smithi132.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.1 MB/s | 320 kB 00:00 2024-04-23T18:23:09.077 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T18:23:09.081 INFO:teuthology.orchestra.run.smithi132.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 298 kB 00:00 2024-04-23T18:23:09.127 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T18:23:09.190 INFO:teuthology.orchestra.run.smithi132.stdout:(24/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 8.3 MB/s | 5.4 MB 00:00 2024-04-23T18:23:09.216 INFO:teuthology.orchestra.run.smithi132.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 741 kB/s | 99 kB 00:00 2024-04-23T18:23:09.249 INFO:teuthology.orchestra.run.smithi132.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 85 kB 00:00 2024-04-23T18:23:09.366 INFO:teuthology.orchestra.run.smithi132.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.4 MB/s | 170 kB 00:00 2024-04-23T18:23:09.466 INFO:teuthology.orchestra.run.smithi132.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 342 kB/s | 34 kB 00:00 2024-04-23T18:23:09.583 INFO:teuthology.orchestra.run.smithi132.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.2 MB/s | 142 kB 00:00 2024-04-23T18:23:09.624 INFO:teuthology.orchestra.run.smithi148.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 502 kB/s | 1.0 MB 00:02 2024-04-23T18:23:09.658 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T18:23:09.676 INFO:teuthology.orchestra.run.smithi132.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 6.7 MB/s | 3.1 MB 00:00 2024-04-23T18:23:09.739 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T18:23:09.768 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T18:23:09.811 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T18:23:09.848 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T18:23:09.899 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T18:23:10.043 INFO:teuthology.orchestra.run.smithi132.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.7 MB/s | 3.5 MB 00:00 2024-04-23T18:23:10.052 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T18:23:10.143 INFO:teuthology.orchestra.run.smithi132.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.4 MB/s | 249 kB 00:00 2024-04-23T18:23:10.152 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T18:23:10.207 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T18:23:10.235 INFO:teuthology.orchestra.run.smithi132.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 539 kB/s | 49 kB 00:00 2024-04-23T18:23:10.279 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T18:23:10.310 INFO:teuthology.orchestra.run.smithi132.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 219 kB/s | 16 kB 00:00 2024-04-23T18:23:10.310 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T18:23:10.347 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T18:23:10.394 INFO:teuthology.orchestra.run.smithi132.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 273 kB 00:00 2024-04-23T18:23:10.394 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T18:23:10.430 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T18:23:10.473 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T18:23:10.515 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T18:23:10.577 INFO:teuthology.orchestra.run.smithi148.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 257 kB/s | 244 kB 00:00 2024-04-23T18:23:10.652 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T18:23:10.704 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T18:23:10.734 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T18:23:10.775 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T18:23:10.844 INFO:teuthology.orchestra.run.smithi132.stdout:(36/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 14 MB/s | 49 MB 00:03 2024-04-23T18:23:10.870 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T18:23:10.886 INFO:teuthology.orchestra.run.smithi132.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 1.5 MB/s | 762 kB 00:00 2024-04-23T18:23:10.900 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T18:23:10.911 INFO:teuthology.orchestra.run.smithi132.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 274 kB/s | 18 kB 00:00 2024-04-23T18:23:10.940 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T18:23:10.966 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T18:23:10.991 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T18:23:10.995 INFO:teuthology.orchestra.run.smithi132.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 5.6 MB/s | 7.4 MB 00:01 2024-04-23T18:23:11.066 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T18:23:11.193 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T18:23:11.231 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T18:23:11.259 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T18:23:11.294 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T18:23:11.332 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T18:23:11.345 INFO:teuthology.orchestra.run.smithi132.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 175 kB/s | 80 kB 00:00 2024-04-23T18:23:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T18:23:11.481 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T18:23:11.512 INFO:teuthology.orchestra.run.smithi132.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 378 kB/s | 195 kB 00:00 2024-04-23T18:23:11.516 INFO:teuthology.orchestra.run.smithi148.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 263 kB/s | 248 kB 00:00 2024-04-23T18:23:11.621 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T18:23:11.646 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T18:23:11.670 INFO:teuthology.orchestra.run.smithi132.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-23T18:23:11.689 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T18:23:11.696 INFO:teuthology.orchestra.run.smithi132.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 722 kB/s | 253 kB 00:00 2024-04-23T18:23:11.722 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T18:23:11.771 INFO:teuthology.orchestra.run.smithi132.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-23T18:23:11.805 INFO:teuthology.orchestra.run.smithi132.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-23T18:23:11.814 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T18:23:11.861 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T18:23:11.872 INFO:teuthology.orchestra.run.smithi132.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-23T18:23:11.891 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T18:23:11.914 INFO:teuthology.orchestra.run.smithi132.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-23T18:23:11.939 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T18:23:11.956 INFO:teuthology.orchestra.run.smithi132.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-04-23T18:23:12.047 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T18:23:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-04-23T18:23:12.101 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T18:23:12.116 INFO:teuthology.orchestra.run.smithi148.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-04-23T18:23:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-04-23T18:23:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-23T18:23:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-04-23T18:23:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-04-23T18:23:12.232 INFO:teuthology.orchestra.run.smithi132.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-23T18:23:12.266 INFO:teuthology.orchestra.run.smithi132.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-04-23T18:23:12.341 INFO:teuthology.orchestra.run.smithi132.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-23T18:23:12.368 INFO:teuthology.orchestra.run.smithi132.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 83 kB/s | 11 kB 00:00 2024-04-23T18:23:12.392 INFO:teuthology.orchestra.run.smithi132.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-04-23T18:23:12.417 INFO:teuthology.orchestra.run.smithi132.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-04-23T18:23:12.459 INFO:teuthology.orchestra.run.smithi132.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-23T18:23:12.484 INFO:teuthology.orchestra.run.smithi132.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-04-23T18:23:12.518 INFO:teuthology.orchestra.run.smithi132.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-04-23T18:23:12.543 INFO:teuthology.orchestra.run.smithi132.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 564 kB/s | 47 kB 00:00 2024-04-23T18:23:12.568 INFO:teuthology.orchestra.run.smithi132.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-23T18:23:12.594 INFO:teuthology.orchestra.run.smithi132.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-23T18:23:12.610 INFO:teuthology.orchestra.run.smithi132.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 526 kB/s | 35 kB 00:00 2024-04-23T18:23:12.741 INFO:teuthology.orchestra.run.smithi148.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 31 kB/s | 19 kB 00:00 2024-04-23T18:23:12.778 INFO:teuthology.orchestra.run.smithi132.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-04-23T18:23:12.820 INFO:teuthology.orchestra.run.smithi132.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-23T18:23:12.836 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T18:23:12.853 INFO:teuthology.orchestra.run.smithi132.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-23T18:23:12.879 INFO:teuthology.orchestra.run.smithi132.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-23T18:23:12.881 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T18:23:12.915 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T18:23:13.017 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T18:23:13.029 INFO:teuthology.orchestra.run.smithi132.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-04-23T18:23:13.074 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T18:23:13.104 INFO:teuthology.orchestra.run.smithi132.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-04-23T18:23:13.162 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T18:23:13.171 INFO:teuthology.orchestra.run.smithi132.stdout:(73/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-04-23T18:23:13.196 INFO:teuthology.orchestra.run.smithi132.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 524 kB/s | 87 kB 00:00 2024-04-23T18:23:13.200 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T18:23:13.238 INFO:teuthology.orchestra.run.smithi132.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-23T18:23:13.263 INFO:teuthology.orchestra.run.smithi132.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-23T18:23:13.297 INFO:teuthology.orchestra.run.smithi132.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-04-23T18:23:13.322 INFO:teuthology.orchestra.run.smithi132.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-04-23T18:23:13.348 INFO:teuthology.orchestra.run.smithi132.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-23T18:23:13.377 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T18:23:13.407 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T18:23:13.465 INFO:teuthology.orchestra.run.smithi132.stdout:(80/152): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-23T18:23:13.484 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T18:23:13.490 INFO:teuthology.orchestra.run.smithi132.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 208 kB/s | 35 kB 00:00 2024-04-23T18:23:13.524 INFO:teuthology.orchestra.run.smithi132.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-23T18:23:13.549 INFO:teuthology.orchestra.run.smithi132.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-23T18:23:13.583 INFO:teuthology.orchestra.run.smithi132.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-23T18:23:13.608 INFO:teuthology.orchestra.run.smithi132.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-04-23T18:23:13.683 INFO:teuthology.orchestra.run.smithi132.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-23T18:23:13.708 INFO:teuthology.orchestra.run.smithi132.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-04-23T18:23:13.734 INFO:teuthology.orchestra.run.smithi132.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-04-23T18:23:13.759 INFO:teuthology.orchestra.run.smithi132.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-23T18:23:13.767 INFO:teuthology.orchestra.run.smithi148.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 168 kB/s | 172 kB 00:01 2024-04-23T18:23:13.784 INFO:teuthology.orchestra.run.smithi132.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-04-23T18:23:13.809 INFO:teuthology.orchestra.run.smithi132.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-04-23T18:23:13.851 INFO:teuthology.orchestra.run.smithi132.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 622 kB/s | 42 kB 00:00 2024-04-23T18:23:13.885 INFO:teuthology.orchestra.run.smithi132.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 586 kB/s | 44 kB 00:00 2024-04-23T18:23:13.927 INFO:teuthology.orchestra.run.smithi132.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-23T18:23:13.952 INFO:teuthology.orchestra.run.smithi132.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 534 kB/s | 36 kB 00:00 2024-04-23T18:23:13.995 INFO:teuthology.orchestra.run.smithi132.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 705 kB/s | 47 kB 00:00 2024-04-23T18:23:14.020 INFO:teuthology.orchestra.run.smithi132.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 305 kB 00:00 2024-04-23T18:23:14.062 INFO:teuthology.orchestra.run.smithi132.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 862 kB/s | 58 kB 00:00 2024-04-23T18:23:14.087 INFO:teuthology.orchestra.run.smithi132.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-23T18:23:14.129 INFO:teuthology.orchestra.run.smithi132.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 951 kB/s | 64 kB 00:00 2024-04-23T18:23:14.138 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T18:23:14.154 INFO:teuthology.orchestra.run.smithi132.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 668 kB/s | 45 kB 00:00 2024-04-23T18:23:14.256 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T18:23:14.372 INFO:teuthology.orchestra.run.smithi132.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2024-04-23T18:23:14.556 INFO:teuthology.orchestra.run.smithi132.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-23T18:23:14.609 INFO:teuthology.orchestra.run.smithi148.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 41 kB/s | 35 kB 00:00 2024-04-23T18:23:14.772 INFO:teuthology.orchestra.run.smithi132.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-04-23T18:23:14.948 INFO:teuthology.orchestra.run.smithi132.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 111 kB/s | 19 kB 00:00 2024-04-23T18:23:15.414 INFO:teuthology.orchestra.run.smithi132.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 360 kB/s | 309 kB 00:00 2024-04-23T18:23:15.500 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T18:23:15.698 INFO:teuthology.orchestra.run.smithi132.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 413 kB/s | 548 kB 00:01 2024-04-23T18:23:15.773 INFO:teuthology.orchestra.run.smithi132.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 72 kB/s | 26 kB 00:00 2024-04-23T18:23:15.794 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T18:23:15.813 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T18:23:15.839 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T18:23:16.142 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T18:23:16.148 INFO:teuthology.orchestra.run.smithi132.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 108 kB/s | 49 kB 00:00 2024-04-23T18:23:16.150 INFO:teuthology.orchestra.run.smithi148.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 644 kB/s | 5.8 MB 00:09 2024-04-23T18:23:16.173 INFO:teuthology.orchestra.run.smithi132.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 168 kB/s | 67 kB 00:00 2024-04-23T18:23:16.176 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:23:16.266 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:23:16.297 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T18:23:16.324 INFO:teuthology.orchestra.run.smithi132.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 863 kB/s | 151 kB 00:00 2024-04-23T18:23:16.334 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T18:23:16.391 INFO:teuthology.orchestra.run.smithi132.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 4.4 MB 00:01 2024-04-23T18:23:16.750 INFO:teuthology.orchestra.run.smithi132.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 121 kB/s | 43 kB 00:00 2024-04-23T18:23:16.775 INFO:teuthology.orchestra.run.smithi132.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 1.2 MB/s | 548 kB 00:00 2024-04-23T18:23:16.808 INFO:teuthology.orchestra.run.smithi132.stdout:(115/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.3 MB/s | 838 kB 00:00 2024-04-23T18:23:16.813 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T18:23:16.834 INFO:teuthology.orchestra.run.smithi132.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-23T18:23:16.837 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:23:16.859 INFO:teuthology.orchestra.run.smithi132.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-23T18:23:16.909 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:23:16.939 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T18:23:17.326 INFO:teuthology.orchestra.run.smithi132.stdout:(118/152): python3-cachetools-4.2.4-1.el9.noarc 56 kB/s | 32 kB 00:00 2024-04-23T18:23:17.351 INFO:teuthology.orchestra.run.smithi132.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 693 kB/s | 358 kB 00:00 2024-04-23T18:23:17.376 INFO:teuthology.orchestra.run.smithi132.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 586 kB/s | 303 kB 00:00 2024-04-23T18:23:17.392 INFO:teuthology.orchestra.run.smithi148.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 202 kB/s | 250 kB 00:01 2024-04-23T18:23:17.485 INFO:teuthology.orchestra.run.smithi132.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 222 kB/s | 35 kB 00:00 2024-04-23T18:23:17.752 INFO:teuthology.orchestra.run.smithi132.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 513 kB/s | 206 kB 00:00 2024-04-23T18:23:17.777 INFO:teuthology.orchestra.run.smithi132.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 495 kB/s | 144 kB 00:00 2024-04-23T18:23:17.828 INFO:teuthology.orchestra.run.smithi132.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-04-23T18:23:17.886 INFO:teuthology.orchestra.run.smithi132.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-04-23T18:23:18.109 INFO:teuthology.orchestra.run.smithi148.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 643 kB/s | 6.0 MB 00:09 2024-04-23T18:23:18.128 INFO:teuthology.orchestra.run.smithi132.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 80 kB/s | 19 kB 00:00 2024-04-23T18:23:18.179 INFO:teuthology.orchestra.run.smithi132.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-04-23T18:23:18.387 INFO:teuthology.orchestra.run.smithi132.stdout:(128/152): python3-jaraco-8.2.1-3.el9.noarch.rp 17 kB/s | 11 kB 00:00 2024-04-23T18:23:18.438 INFO:teuthology.orchestra.run.smithi132.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 1.9 MB/s | 2.0 MB 00:01 2024-04-23T18:23:18.567 INFO:teuthology.orchestra.run.smithi148.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 189 kB/s | 222 kB 00:01 2024-04-23T18:23:18.821 INFO:teuthology.orchestra.run.smithi132.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 1.6 MB/s | 1.0 MB 00:00 2024-04-23T18:23:18.885 INFO:teuthology.orchestra.run.smithi148.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 100 kB/s | 77 kB 00:00 2024-04-23T18:23:19.263 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:23:19.284 INFO:teuthology.orchestra.run.smithi148.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 59 kB/s | 42 kB 00:00 2024-04-23T18:23:19.294 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:23:19.380 INFO:teuthology.orchestra.run.smithi132.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 47 kB/s | 46 kB 00:00 2024-04-23T18:23:19.405 INFO:teuthology.orchestra.run.smithi132.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 81 kB/s | 79 kB 00:00 2024-04-23T18:23:19.455 INFO:teuthology.orchestra.run.smithi132.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-23T18:23:19.480 INFO:teuthology.orchestra.run.smithi132.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 219 kB/s | 16 kB 00:00 2024-04-23T18:23:19.506 INFO:teuthology.orchestra.run.smithi132.stdout:(135/152): python3-natsort-7.1.1-5.el9.noarch.r 84 kB/s | 58 kB 00:00 2024-04-23T18:23:19.531 INFO:teuthology.orchestra.run.smithi132.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-23T18:23:19.556 INFO:teuthology.orchestra.run.smithi132.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-04-23T18:23:19.631 INFO:teuthology.orchestra.run.smithi132.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 245 kB/s | 31 kB 00:00 2024-04-23T18:23:19.657 INFO:teuthology.orchestra.run.smithi132.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-04-23T18:23:19.682 INFO:teuthology.orchestra.run.smithi132.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-23T18:23:19.835 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:23:19.872 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:23:19.957 INFO:teuthology.orchestra.run.smithi132.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 110 kB/s | 36 kB 00:00 2024-04-23T18:23:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:23:19.964 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T18:23:19.964 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T18:23:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:19.991 INFO:teuthology.orchestra.run.smithi132.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 153 kB/s | 51 kB 00:00 2024-04-23T18:23:19.999 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:23:20.012 INFO:teuthology.orchestra.run.smithi148.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 239 kB/s | 269 kB 00:01 2024-04-23T18:23:20.016 INFO:teuthology.orchestra.run.smithi132.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 687 kB/s | 230 kB 00:00 2024-04-23T18:23:20.108 INFO:teuthology.orchestra.run.smithi132.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 596 kB/s | 90 kB 00:00 2024-04-23T18:23:20.133 INFO:teuthology.orchestra.run.smithi132.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-23T18:23:20.204 INFO:teuthology.orchestra.run.smithi148.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 173 kB/s | 159 kB 00:00 2024-04-23T18:23:20.334 INFO:teuthology.orchestra.run.smithi132.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 63 kB/s | 20 kB 00:00 2024-04-23T18:23:20.359 INFO:teuthology.orchestra.run.smithi132.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 764 kB/s | 192 kB 00:00 2024-04-23T18:23:20.651 INFO:teuthology.orchestra.run.smithi132.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 3.1 MB/s | 1.6 MB 00:00 2024-04-23T18:23:20.793 INFO:teuthology.orchestra.run.smithi132.stdout:(149/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 7.3 MB/s | 3.1 MB 00:00 2024-04-23T18:23:21.004 INFO:teuthology.orchestra.run.smithi148.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 64 kB/s | 51 kB 00:00 2024-04-23T18:23:21.018 INFO:teuthology.orchestra.run.smithi132.stdout:(150/152): librados2-19.0.0-3156.gd69f13d8.el7. 5.0 MB/s | 3.4 MB 00:00 2024-04-23T18:23:21.197 INFO:teuthology.orchestra.run.smithi148.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 236 kB/s | 279 kB 00:01 2024-04-23T18:23:21.718 INFO:teuthology.orchestra.run.smithi132.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-04-23T18:23:21.838 INFO:teuthology.orchestra.run.smithi148.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 64 kB/s | 54 kB 00:00 2024-04-23T18:23:21.968 INFO:teuthology.orchestra.run.smithi132.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-04-23T18:23:21.971 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:23:21.972 INFO:teuthology.orchestra.run.smithi132.stdout:Total 13 MB/s | 222 MB 00:17 2024-04-23T18:23:22.596 INFO:teuthology.orchestra.run.smithi148.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 55 kB/s | 42 kB 00:00 2024-04-23T18:23:23.082 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-04-23T18:23:23.224 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-04-23T18:23:23.225 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-04-23T18:23:23.380 INFO:teuthology.orchestra.run.smithi148.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 56 kB/s | 44 kB 00:00 2024-04-23T18:23:23.447 INFO:teuthology.orchestra.run.smithi148.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 591 kB/s | 5.1 MB 00:08 2024-04-23T18:23:24.205 INFO:teuthology.orchestra.run.smithi148.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 47 kB/s | 36 kB 00:00 2024-04-23T18:23:24.506 INFO:teuthology.orchestra.run.smithi148.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 196 kB/s | 221 kB 00:01 2024-04-23T18:23:24.986 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-04-23T18:23:24.987 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-04-23T18:23:25.039 INFO:teuthology.orchestra.run.smithi148.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 57 kB/s | 47 kB 00:00 2024-04-23T18:23:25.948 INFO:teuthology.orchestra.run.smithi148.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 64 kB/s | 58 kB 00:00 2024-04-23T18:23:26.006 INFO:teuthology.orchestra.run.smithi148.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 207 kB/s | 305 kB 00:01 2024-04-23T18:23:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 72 kB/s | 64 kB 00:00 2024-04-23T18:23:26.918 INFO:teuthology.orchestra.run.smithi148.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 97 kB/s | 94 kB 00:00 2024-04-23T18:23:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-04-23T18:23:27.541 INFO:teuthology.orchestra.run.smithi148.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-04-23T18:23:27.641 INFO:teuthology.orchestra.run.smithi148.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 60 kB/s | 45 kB 00:00 2024-04-23T18:23:28.358 INFO:teuthology.orchestra.run.smithi148.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 671 kB/s | 548 kB 00:00 2024-04-23T18:23:28.383 INFO:teuthology.orchestra.run.smithi148.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 416 kB/s | 309 kB 00:00 2024-04-23T18:23:28.458 INFO:teuthology.orchestra.run.smithi148.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-04-23T18:23:28.558 INFO:teuthology.orchestra.run.smithi148.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-23T18:23:28.667 INFO:teuthology.orchestra.run.smithi148.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-04-23T18:23:28.801 INFO:teuthology.orchestra.run.smithi148.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 506 kB/s | 67 kB 00:00 2024-04-23T18:23:28.976 INFO:teuthology.orchestra.run.smithi148.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 7.5 MB/s | 4.4 MB 00:00 2024-04-23T18:23:29.001 INFO:teuthology.orchestra.run.smithi148.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 754 kB/s | 151 kB 00:00 2024-04-23T18:23:29.118 INFO:teuthology.orchestra.run.smithi148.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 838 kB 00:00 2024-04-23T18:23:29.185 INFO:teuthology.orchestra.run.smithi148.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-04-23T18:23:29.236 INFO:teuthology.orchestra.run.smithi148.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 370 kB/s | 43 kB 00:00 2024-04-23T18:23:29.294 INFO:teuthology.orchestra.run.smithi148.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-23T18:23:29.361 INFO:teuthology.orchestra.run.smithi148.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 119 kB/s | 15 kB 00:00 2024-04-23T18:23:29.445 INFO:teuthology.orchestra.run.smithi148.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-04-23T18:23:29.504 INFO:teuthology.orchestra.run.smithi148.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-04-23T18:23:29.612 INFO:teuthology.orchestra.run.smithi148.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-04-23T18:23:29.637 INFO:teuthology.orchestra.run.smithi148.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 264 kB/s | 35 kB 00:00 2024-04-23T18:23:29.830 INFO:teuthology.orchestra.run.smithi148.stdout:(121/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-23T18:23:29.980 INFO:teuthology.orchestra.run.smithi148.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 963 kB/s | 144 kB 00:00 2024-04-23T18:23:30.072 INFO:teuthology.orchestra.run.smithi148.stdout:(123/152): python3-google-auth-2.29.0-1.el9.noa 448 kB/s | 206 kB 00:00 2024-04-23T18:23:30.097 INFO:teuthology.orchestra.run.smithi148.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-04-23T18:23:30.197 INFO:teuthology.orchestra.run.smithi148.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-04-23T18:23:30.222 INFO:teuthology.orchestra.run.smithi148.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-04-23T18:23:30.323 INFO:teuthology.orchestra.run.smithi148.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2024-04-23T18:23:30.356 INFO:teuthology.orchestra.run.smithi148.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 147 kB/s | 20 kB 00:00 2024-04-23T18:23:30.507 INFO:teuthology.orchestra.run.smithi148.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 309 kB/s | 46 kB 00:00 2024-04-23T18:23:30.582 INFO:teuthology.orchestra.run.smithi148.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-04-23T18:23:30.657 INFO:teuthology.orchestra.run.smithi148.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 525 kB/s | 79 kB 00:00 2024-04-23T18:23:30.716 INFO:teuthology.orchestra.run.smithi148.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 432 kB/s | 58 kB 00:00 2024-04-23T18:23:30.802 INFO:teuthology.orchestra.run.smithi148.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-04-23T18:23:30.841 INFO:teuthology.orchestra.run.smithi148.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-04-23T18:23:30.912 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-04-23T18:23:30.933 INFO:teuthology.orchestra.run.smithi148.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 674 kB/s | 90 kB 00:00 2024-04-23T18:23:30.967 INFO:teuthology.orchestra.run.smithi148.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 225 kB/s | 28 kB 00:00 2024-04-23T18:23:31.006 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T18:23:31.044 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T18:23:31.067 INFO:teuthology.orchestra.run.smithi148.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 230 kB/s | 31 kB 00:00 2024-04-23T18:23:31.106 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T18:23:31.109 INFO:teuthology.orchestra.run.smithi148.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-04-23T18:23:31.140 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T18:23:31.165 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T18:23:31.201 INFO:teuthology.orchestra.run.smithi148.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2024-04-23T18:23:31.209 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T18:23:31.234 INFO:teuthology.orchestra.run.smithi148.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 286 kB/s | 36 kB 00:00 2024-04-23T18:23:31.310 INFO:teuthology.orchestra.run.smithi148.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-04-23T18:23:31.352 INFO:teuthology.orchestra.run.smithi148.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-23T18:23:31.427 INFO:teuthology.orchestra.run.smithi148.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 765 kB/s | 90 kB 00:00 2024-04-23T18:23:31.478 INFO:teuthology.orchestra.run.smithi148.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-23T18:23:31.525 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:23:31.525 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-04-23T18:23:31.525 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-04-23T18:23:31.525 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-04-23T18:23:31.525 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-04-23T18:23:31.525 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-04-23T18:23:31.525 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-04-23T18:23:31.525 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-04-23T18:23:31.526 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:31.544 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T18:23:31.578 INFO:teuthology.orchestra.run.smithi148.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 133 kB/s | 20 kB 00:00 2024-04-23T18:23:31.603 INFO:teuthology.orchestra.run.smithi148.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-04-23T18:23:31.658 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:23:31.711 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:23:31.730 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:23:31.730 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:31.730 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T18:23:31.730 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:31.745 INFO:teuthology.orchestra.run.smithi148.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-23T18:23:31.780 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:23:31.817 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:23:31.909 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:23:31.938 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:23:32.084 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T18:23:32.135 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T18:23:32.154 INFO:teuthology.orchestra.run.smithi148.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 7.7 MB/s | 3.1 MB 00:00 2024-04-23T18:23:32.163 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:23:32.292 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:23:32.315 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:23:32.321 INFO:teuthology.orchestra.run.smithi148.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-04-23T18:23:32.377 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:23:32.399 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:23:32.404 INFO:teuthology.orchestra.run.smithi148.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-04-23T18:23:32.419 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:23:32.607 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:23:32.640 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:23:32.747 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T18:23:32.834 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T18:23:32.846 INFO:teuthology.orchestra.run.smithi148.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 2.7 MB/s | 3.4 MB 00:01 2024-04-23T18:23:32.873 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T18:23:32.907 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T18:23:32.972 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T18:23:33.007 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T18:23:33.050 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T18:23:33.082 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T18:23:33.111 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T18:23:33.169 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T18:23:33.224 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T18:23:33.231 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:23:33.255 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T18:23:33.257 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:23:33.257 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:33.257 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T18:23:33.257 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:23:33.257 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:23:33.258 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:33.290 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T18:23:33.366 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T18:23:33.401 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T18:23:33.444 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T18:23:33.473 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T18:23:33.490 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:23:33.514 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:23:33.514 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:33.514 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T18:23:33.514 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:23:33.515 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:23:33.515 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:33.542 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T18:23:33.596 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T18:23:33.670 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T18:23:33.801 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T18:23:33.857 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T18:23:33.966 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:23:33.990 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:23:33.990 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:33.990 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T18:23:33.990 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:23:33.990 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:23:33.990 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:34.104 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T18:23:34.152 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T18:23:34.209 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T18:23:34.372 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T18:23:34.394 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T18:23:34.442 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T18:23:34.502 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T18:23:34.520 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T18:23:34.609 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:23:34.682 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T18:23:34.725 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T18:23:34.758 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:23:34.762 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T18:23:34.800 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T18:23:34.804 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T18:23:34.832 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T18:23:34.860 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T18:23:34.960 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T18:23:35.015 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T18:23:35.069 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T18:23:35.134 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T18:23:35.166 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T18:23:35.203 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T18:23:35.249 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T18:23:35.285 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T18:23:35.304 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:23:35.320 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T18:23:35.333 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:23:35.362 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T18:23:35.365 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T18:23:35.427 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T18:23:35.457 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:23:35.497 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T18:23:35.542 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T18:23:35.573 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T18:23:35.614 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T18:23:35.710 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T18:23:35.740 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T18:23:35.771 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T18:23:35.797 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T18:23:35.822 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T18:23:35.896 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T18:23:36.023 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T18:23:36.055 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T18:23:36.082 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T18:23:36.117 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T18:23:36.156 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T18:23:36.212 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T18:23:36.242 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T18:23:36.277 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T18:23:36.310 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T18:23:36.352 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T18:23:36.386 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T18:23:36.472 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T18:23:36.494 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:23:36.534 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:23:36.684 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T18:23:36.723 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T18:23:36.770 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T18:23:36.867 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T18:23:36.916 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T18:23:37.497 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:23:37.526 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:23:37.629 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:23:37.630 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T18:23:37.680 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T18:23:37.714 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T18:23:37.798 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T18:23:37.811 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T18:23:37.828 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:23:37.851 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:23:37.851 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:37.851 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T18:23:37.851 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:23:37.851 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:23:37.851 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:37.863 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T18:23:37.895 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:23:37.906 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:23:37.950 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T18:23:38.005 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T18:23:38.009 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T18:23:38.052 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T18:23:38.176 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T18:23:38.206 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T18:23:38.282 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T18:23:38.921 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T18:23:38.975 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T18:23:39.006 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:23:39.028 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:23:39.028 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:39.028 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T18:23:39.028 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:23:39.029 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:23:39.029 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:39.036 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T18:23:39.073 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:23:39.096 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:23:39.096 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:39.096 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T18:23:39.096 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:39.140 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:23:39.163 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:23:39.163 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:39.163 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T18:23:39.163 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:23:39.163 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:23:39.163 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:39.475 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:23:39.498 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:23:39.498 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:39.498 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T18:23:39.498 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:23:39.498 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:23:39.498 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:40.250 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T18:23:40.344 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T18:23:40.371 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T18:23:40.405 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T18:23:40.715 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T18:23:40.759 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:23:40.833 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:23:40.864 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T18:23:40.901 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T18:23:41.380 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T18:23:41.413 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:23:41.467 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:23:41.489 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T18:23:43.782 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:23:43.820 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:23:44.103 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T18:23:44.144 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T18:23:44.186 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T18:23:44.217 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T18:23:44.245 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T18:23:44.358 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:23:44.398 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:23:44.433 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T18:23:44.474 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T18:23:44.489 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:23:44.489 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T18:23:44.489 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T18:23:44.490 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:23:44.496 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T18:23:44.524 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:23:45.123 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:23:45.168 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:23:45.188 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:23:46.035 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T18:23:46.123 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T18:23:46.156 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T18:23:46.238 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T18:23:46.277 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T18:23:46.277 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:23:46.361 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:23:46.394 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:23:46.394 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:23:46.487 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:23:46.559 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:23:47.820 INFO:teuthology.orchestra.run.smithi148.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 625 kB/s | 16 MB 00:26 2024-04-23T18:23:47.823 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:23:47.823 INFO:teuthology.orchestra.run.smithi148.stdout:Total 3.4 MB/s | 222 MB 01:04 2024-04-23T18:23:48.869 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T18:23:49.012 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T18:23:49.012 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T18:23:50.775 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T18:23:50.776 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T18:23:52.516 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:23:52.517 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T18:23:52.517 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T18:23:52.517 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T18:23:52.517 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T18:23:52.517 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T18:23:52.517 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T18:23:52.517 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T18:23:52.517 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:23:52.517 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:23:52.517 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:23:52.517 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T18:23:52.517 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T18:23:52.518 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:23:52.518 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T18:23:52.518 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T18:23:52.518 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T18:23:52.518 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T18:23:52.518 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T18:23:52.518 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T18:23:52.518 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T18:23:52.518 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T18:23:52.518 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T18:23:52.518 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T18:23:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T18:23:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T18:23:52.521 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T18:23:52.522 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T18:23:52.523 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T18:23:52.524 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:23:52.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T18:23:52.526 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T18:23:52.527 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T18:23:52.528 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T18:23:52.528 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T18:23:52.528 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T18:23:52.528 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T18:23:52.528 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T18:23:52.528 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T18:23:52.528 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:23:52.528 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T18:23:54.592 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:23:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:54.593 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-04-23T18:23:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T18:23:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T18:23:54.593 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-23T18:23:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T18:23:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T18:23:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T18:23:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T18:23:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:23:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T18:23:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T18:23:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T18:23:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T18:23:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T18:23:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T18:23:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T18:23:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T18:23:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T18:23:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T18:23:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:23:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T18:23:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T18:23:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T18:23:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T18:23:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T18:23:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T18:23:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T18:23:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:23:54.604 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-23T18:23:54.996 DEBUG:teuthology.parallel:result is None 2024-04-23T18:23:55.938 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:23:55.938 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /sys 2024-04-23T18:23:55.938 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /proc 2024-04-23T18:23:55.939 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /mnt 2024-04-23T18:23:55.939 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /var/tmp 2024-04-23T18:23:55.939 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /home 2024-04-23T18:23:55.939 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /root 2024-04-23T18:23:55.939 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /tmp 2024-04-23T18:23:55.939 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:23:56.108 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:23:56.123 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:23:56.123 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:56.124 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T18:23:56.124 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:23:56.734 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T18:23:56.814 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T18:23:56.861 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T18:23:56.914 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T18:23:56.948 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T18:23:56.974 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T18:23:57.017 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T18:23:57.346 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T18:23:57.442 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:23:57.563 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:23:57.585 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:23:57.586 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:23:57.611 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:23:57.611 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:57.611 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T18:23:57.611 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:23:57.611 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:23:57.611 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:23:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:23:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:23:57.836 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:23:57.858 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:23:57.859 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:57.859 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T18:23:57.859 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:23:57.859 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:23:57.859 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:23:57.871 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T18:23:57.927 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T18:23:57.955 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:23:58.049 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:23:58.073 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:23:58.135 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:23:58.157 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:23:58.179 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:23:58.304 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:23:58.326 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:23:58.326 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:23:58.326 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T18:23:58.326 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:23:58.326 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:23:58.327 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:23:58.374 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:23:58.406 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:23:58.425 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T18:23:58.473 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T18:23:58.513 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T18:23:58.528 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T18:23:58.608 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T18:23:58.639 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T18:23:58.673 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T18:23:58.721 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T18:23:58.745 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T18:23:58.770 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T18:23:58.789 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T18:23:58.832 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T18:23:58.839 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T18:23:58.865 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T18:23:58.893 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T18:23:58.928 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:23:58.961 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T18:23:59.015 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T18:23:59.046 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T18:23:59.058 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:23:59.081 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T18:23:59.096 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T18:23:59.128 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T18:23:59.156 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T18:23:59.165 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T18:23:59.200 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T18:23:59.244 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T18:23:59.272 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T18:23:59.341 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T18:23:59.387 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T18:23:59.460 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T18:23:59.525 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:23:59.555 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:23:59.587 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T18:23:59.594 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T18:23:59.655 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T18:23:59.657 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T18:23:59.695 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:24:00.174 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T18:24:00.276 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T18:24:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T18:24:00.357 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T18:24:00.395 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T18:24:00.436 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T18:24:00.582 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T18:24:00.647 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T18:24:00.702 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T18:24:00.730 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:24:00.763 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:24:00.774 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T18:24:00.815 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T18:24:00.860 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T18:24:00.906 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T18:24:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T18:24:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T18:24:01.027 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T18:24:01.162 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T18:24:01.207 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T18:24:01.238 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T18:24:01.279 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T18:24:01.381 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T18:24:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T18:24:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T18:24:01.495 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T18:24:01.520 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T18:24:01.595 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T18:24:01.720 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T18:24:01.726 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:24:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:24:01.760 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T18:24:01.788 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T18:24:01.822 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T18:24:01.858 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:24:01.870 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T18:24:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T18:24:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T18:24:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T18:24:02.016 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T18:24:02.018 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T18:24:02.041 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:24:02.057 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T18:24:02.065 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:24:02.065 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:02.065 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T18:24:02.065 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:24:02.065 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:24:02.065 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:24:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T18:24:02.108 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:24:02.119 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:24:02.179 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T18:24:02.217 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T18:24:02.221 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T18:24:02.262 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T18:24:02.265 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T18:24:02.308 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T18:24:02.405 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T18:24:02.463 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T18:24:03.177 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T18:24:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T18:24:03.227 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T18:24:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:24:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:24:03.250 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:03.250 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T18:24:03.251 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:24:03.251 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:24:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:24:03.269 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T18:24:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:24:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:24:03.318 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:03.318 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T18:24:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:24:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:24:03.379 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T18:24:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:24:03.394 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:03.394 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T18:24:03.394 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:24:03.394 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:24:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:24:03.434 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T18:24:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T18:24:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T18:24:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:24:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:24:03.719 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:03.720 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T18:24:03.720 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:24:03.720 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:24:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:24:03.732 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T18:24:03.760 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T18:24:03.838 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T18:24:04.468 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T18:24:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T18:24:05.790 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T18:24:05.872 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T18:24:05.900 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T18:24:05.933 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T18:24:06.244 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T18:24:06.296 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:24:06.386 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:24:06.417 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T18:24:06.453 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T18:24:06.933 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T18:24:06.957 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:24:07.028 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:24:07.051 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T18:24:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T18:24:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T18:24:08.390 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T18:24:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T18:24:08.450 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T18:24:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T18:24:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T18:24:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T18:24:09.289 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:24:09.322 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:24:09.347 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:24:09.383 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:24:09.402 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:24:09.861 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:24:09.900 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:24:09.975 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:24:09.975 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T18:24:09.975 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T18:24:09.975 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:10.010 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:24:10.227 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T18:24:10.322 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T18:24:10.363 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T18:24:10.445 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T18:24:10.476 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T18:24:10.476 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:24:10.568 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:24:10.593 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:24:10.593 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:24:10.668 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:24:10.733 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:24:16.683 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T18:24:16.684 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T18:24:16.685 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T18:24:16.697 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T18:24:16.697 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T18:24:16.697 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T18:24:16.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T18:24:16.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T18:24:16.700 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T18:24:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T18:24:16.702 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T18:24:16.703 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T18:24:16.704 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:24:16.705 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T18:24:18.766 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:24:18.767 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:24:18.767 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2024-04-23T18:24:18.767 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.767 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.767 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T18:24:18.767 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T18:24:18.767 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-04-23T18:24:18.767 INFO:teuthology.orchestra.run.smithi132.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T18:24:18.767 INFO:teuthology.orchestra.run.smithi132.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T18:24:18.767 INFO:teuthology.orchestra.run.smithi132.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T18:24:18.767 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T18:24:18.767 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.768 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T18:24:18.769 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.770 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T18:24:18.771 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T18:24:18.772 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T18:24:18.773 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T18:24:18.774 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T18:24:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T18:24:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T18:24:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T18:24:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T18:24:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:24:18.777 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-04-23T18:24:19.191 DEBUG:teuthology.parallel:result is None 2024-04-23T18:24:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:24:21.376 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-04-23T18:24:21.376 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-04-23T18:24:21.376 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-04-23T18:24:21.376 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-04-23T18:24:21.376 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-04-23T18:24:21.376 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-04-23T18:24:21.376 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-04-23T18:24:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:21.546 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:24:21.561 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:24:21.561 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:21.561 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T18:24:21.561 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:24:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:24:23.066 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:23.066 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T18:24:23.067 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:24:23.067 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:24:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:23.291 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:24:23.313 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:24:23.313 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:23.313 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T18:24:23.313 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:24:23.313 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:24:23.314 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:23.759 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:24:23.780 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:24:23.780 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:23.780 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T18:24:23.781 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:24:23.781 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:24:23.781 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T18:24:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T18:24:24.000 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T18:24:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T18:24:24.225 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T18:24:24.302 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T18:24:24.404 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:24:24.538 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:24:24.577 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T18:24:24.607 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T18:24:24.635 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T18:24:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:24:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:24:25.074 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T18:24:25.135 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T18:24:25.173 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:24:26.211 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:24:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:24:27.233 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:24:27.295 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:24:27.395 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:24:27.573 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T18:24:27.611 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:24:27.633 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:24:27.633 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:27.633 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T18:24:27.633 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:24:27.633 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:24:27.633 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:24:27.689 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:24:27.787 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T18:24:27.835 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T18:24:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T18:24:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:24:28.810 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:24:28.810 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:28.811 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T18:24:28.811 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:24:28.811 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:24:28.811 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:28.856 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:24:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:24:28.878 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:28.878 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T18:24:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:28.932 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:24:28.954 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:24:28.954 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:28.955 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T18:24:28.955 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:24:28.955 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:24:28.955 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:24:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:24:29.280 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:24:29.280 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T18:24:29.280 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:24:29.280 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:24:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:33.878 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T18:24:33.919 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T18:24:33.969 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T18:24:34.001 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T18:24:34.028 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T18:24:34.220 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T18:24:34.266 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T18:24:34.296 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T18:24:34.892 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:24:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:24:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:24:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T18:24:35.889 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T18:24:35.930 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T18:24:36.011 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T18:24:36.044 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T18:24:36.044 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:24:36.136 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:24:36.177 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:24:36.178 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:24:36.270 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:24:36.334 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:24:42.347 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:24:42.347 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T18:24:42.347 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T18:24:42.348 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T18:24:42.348 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T18:24:42.348 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T18:24:42.348 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T18:24:42.348 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T18:24:42.348 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:24:42.348 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:24:42.348 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:24:42.348 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T18:24:42.348 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T18:24:42.348 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:24:42.349 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T18:24:42.349 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T18:24:42.349 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T18:24:42.349 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T18:24:42.349 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T18:24:42.349 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T18:24:42.349 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T18:24:42.349 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T18:24:42.349 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T18:24:42.349 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T18:24:42.349 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T18:24:42.349 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T18:24:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T18:24:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T18:24:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T18:24:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T18:24:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T18:24:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T18:24:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T18:24:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T18:24:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T18:24:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T18:24:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T18:24:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T18:24:42.351 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T18:24:42.351 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T18:24:42.351 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T18:24:42.351 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T18:24:42.351 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T18:24:42.351 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T18:24:42.351 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T18:24:42.351 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T18:24:42.351 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T18:24:42.351 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T18:24:42.351 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T18:24:42.352 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T18:24:42.352 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T18:24:42.352 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T18:24:42.352 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T18:24:42.352 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T18:24:42.352 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T18:24:42.352 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T18:24:42.352 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T18:24:42.352 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T18:24:42.352 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T18:24:42.352 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T18:24:42.352 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T18:24:42.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T18:24:42.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T18:24:42.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T18:24:42.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T18:24:42.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T18:24:42.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T18:24:42.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T18:24:42.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T18:24:42.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T18:24:42.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T18:24:42.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T18:24:42.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T18:24:42.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T18:24:42.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T18:24:42.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T18:24:42.357 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T18:24:42.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T18:24:42.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T18:24:42.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T18:24:42.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T18:24:42.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T18:24:42.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T18:24:42.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T18:24:42.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T18:24:42.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T18:24:42.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T18:24:42.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T18:24:42.358 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T18:24:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T18:24:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T18:24:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T18:24:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T18:24:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T18:24:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T18:24:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T18:24:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T18:24:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T18:24:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T18:24:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T18:24:42.359 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T18:24:42.360 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T18:24:42.360 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T18:24:42.360 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T18:24:42.360 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T18:24:42.360 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:24:42.360 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T18:24:44.424 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:24:44.424 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:44.424 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-04-23T18:24:44.424 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:44.425 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.426 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T18:24:44.427 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T18:24:44.428 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T18:24:44.429 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T18:24:44.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T18:24:44.431 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T18:24:44.432 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.433 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T18:24:44.434 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T18:24:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:24:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T18:24:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T18:24:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T18:24:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T18:24:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T18:24:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T18:24:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T18:24:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T18:24:44.436 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T18:24:44.826 DEBUG:teuthology.parallel:result is None 2024-04-23T18:24:44.827 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:24:44.990 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T18:24:45.016 INFO:teuthology.orchestra.run.smithi132.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T18:24:45.017 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T18:24:45.017 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T18:24:45.019 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:24:45.245 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T18:24:45.266 INFO:teuthology.orchestra.run.smithi148.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T18:24:45.266 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T18:24:45.267 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T18:24:45.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:24:45.405 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T18:24:45.431 INFO:teuthology.orchestra.run.smithi157.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T18:24:45.431 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T18:24:45.431 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T18:24:45.433 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T18:24:45.433 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:24:45.433 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T18:24:45.465 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T18:24:45.465 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T18:24:45.491 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-23T18:24:45.491 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T18:24:45.521 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T18:24:45.521 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:24:45.521 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T18:24:45.549 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T18:24:45.615 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T18:24:45.615 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T18:24:45.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T18:24:45.702 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-23T18:24:45.702 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T18:24:45.729 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T18:24:45.793 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T18:24:45.794 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:24:45.794 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T18:24:45.821 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T18:24:45.888 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T18:24:45.888 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T18:24:45.911 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T18:24:45.975 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-23T18:24:45.975 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T18:24:45.999 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T18:24:46.065 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T18:24:46.065 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:24:46.066 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T18:24:46.093 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T18:24:46.159 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T18:24:46.159 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T18:24:46.182 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T18:24:46.245 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-23T18:24:46.245 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T18:24:46.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T18:24:46.333 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-23T18:24:46.438 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-23T18:24:46.439 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:24:46.439 INFO:tasks.cephadm:Cluster fsid is c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:24:46.439 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-23T18:24:46.440 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.132', 'mon.b': '172.21.15.148', 'mon.c': '172.21.15.157'} 2024-04-23T18:24:46.440 INFO:tasks.cephadm:First mon is mon.a on smithi132 2024-04-23T18:24:46.440 INFO:tasks.cephadm:First mgr is x 2024-04-23T18:24:46.440 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-23T18:24:46.440 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname $(hostname -s) 2024-04-23T18:24:46.468 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname $(hostname -s) 2024-04-23T18:24:46.491 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2024-04-23T18:24:46.516 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-23T18:24:46.517 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:24:46.733 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-23T18:24:46.871 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:24:46.873 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-23T18:24:46.873 DEBUG:teuthology.orchestra.run.smithi132:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T18:24:47.308 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 18:24 /home/ubuntu/cephtest/cephadm 2024-04-23T18:24:47.308 DEBUG:teuthology.orchestra.run.smithi148:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T18:24:47.737 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 18:24 /home/ubuntu/cephtest/cephadm 2024-04-23T18:24:47.738 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T18:24:48.487 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 18:24 /home/ubuntu/cephtest/cephadm 2024-04-23T18:24:48.487 DEBUG:teuthology.orchestra.run.smithi132:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T18:24:48.506 DEBUG:teuthology.orchestra.run.smithi148:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T18:24:48.526 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T18:24:48.561 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-23T18:24:48.561 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T18:24:48.563 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T18:24:48.568 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T18:24:50.225 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:24:50.228 INFO:teuthology.orchestra.run.smithi132.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:24:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:25:19.662 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:25:19.662 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T18:25:19.662 INFO:teuthology.orchestra.run.smithi132.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T18:25:19.662 INFO:teuthology.orchestra.run.smithi132.stdout: "repo_digests": [ 2024-04-23T18:25:19.662 INFO:teuthology.orchestra.run.smithi132.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T18:25:19.662 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-04-23T18:25:19.662 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:25:41.868 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-23T18:25:41.868 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T18:25:41.868 INFO:teuthology.orchestra.run.smithi148.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T18:25:41.869 INFO:teuthology.orchestra.run.smithi148.stdout: "repo_digests": [ 2024-04-23T18:25:41.869 INFO:teuthology.orchestra.run.smithi148.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T18:25:41.869 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-23T18:25:41.869 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-23T18:25:48.553 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-23T18:25:48.553 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T18:25:48.553 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T18:25:48.553 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2024-04-23T18:25:48.553 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T18:25:48.553 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2024-04-23T18:25:48.553 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-23T18:25:48.577 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph 2024-04-23T18:25:48.610 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph 2024-04-23T18:25:48.641 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2024-04-23T18:25:48.667 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /etc/ceph 2024-04-23T18:25:48.694 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /etc/ceph 2024-04-23T18:25:48.721 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2024-04-23T18:25:48.745 INFO:tasks.cephadm:Writing seed config... 2024-04-23T18:25:48.746 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-23T18:25:48.746 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-23T18:25:48.746 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-23T18:25:48.746 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-23T18:25:48.746 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-23T18:25:48.747 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-23T18:25:48.748 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-23T18:25:48.749 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-23T18:25:48.749 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-23T18:25:48.749 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-23T18:25:48.749 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-23T18:25:48.749 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-23T18:25:48.749 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-23T18:25:48.749 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-23T18:25:48.749 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-23T18:25:48.749 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-23T18:25:48.749 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-23T18:25:48.749 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-23T18:25:48.750 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:25:48.750 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-23T18:25:48.768 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c34642b4-019e-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-23T18:25:48.768 DEBUG:teuthology.orchestra.run.smithi132:mon.a> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.a.service 2024-04-23T18:25:48.810 DEBUG:teuthology.orchestra.run.smithi132:mgr.x> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.x.service 2024-04-23T18:25:48.854 INFO:tasks.cephadm:Bootstrapping... 2024-04-23T18:25:48.854 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid c34642b4-019e-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.132 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:25:49.046 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:25:49.046 INFO:teuthology.orchestra.run.smithi132.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'c34642b4-019e-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.132', '--skip-admin-label'] 2024-04-23T18:25:49.047 INFO:teuthology.orchestra.run.smithi132.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-23T18:25:49.047 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying podman|docker is present... 2024-04-23T18:25:49.072 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T18:25:49.073 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying lvm2 is present... 2024-04-23T18:25:49.073 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying time synchronization is in place... 2024-04-23T18:25:49.080 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T18:25:49.080 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T18:25:49.086 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T18:25:49.086 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout inactive 2024-04-23T18:25:49.092 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout enabled 2024-04-23T18:25:49.098 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout active 2024-04-23T18:25:49.098 INFO:teuthology.orchestra.run.smithi132.stdout:Unit chronyd.service is enabled and running 2024-04-23T18:25:49.098 INFO:teuthology.orchestra.run.smithi132.stdout:Repeating the final host check... 2024-04-23T18:25:49.122 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T18:25:49.122 INFO:teuthology.orchestra.run.smithi132.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-23T18:25:49.122 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl is present 2024-04-23T18:25:49.122 INFO:teuthology.orchestra.run.smithi132.stdout:lvcreate is present 2024-04-23T18:25:49.128 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T18:25:49.128 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T18:25:49.134 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T18:25:49.134 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout inactive 2024-04-23T18:25:49.140 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout enabled 2024-04-23T18:25:49.146 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout active 2024-04-23T18:25:49.146 INFO:teuthology.orchestra.run.smithi132.stdout:Unit chronyd.service is enabled and running 2024-04-23T18:25:49.146 INFO:teuthology.orchestra.run.smithi132.stdout:Host looks OK 2024-04-23T18:25:49.146 INFO:teuthology.orchestra.run.smithi132.stdout:Cluster fsid: c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:25:49.147 INFO:teuthology.orchestra.run.smithi132.stdout:Acquiring lock 139672103095456 on /run/cephadm/c34642b4-019e-11ef-bc93-c7b262605968.lock 2024-04-23T18:25:49.147 INFO:teuthology.orchestra.run.smithi132.stdout:Lock 139672103095456 acquired on /run/cephadm/c34642b4-019e-11ef-bc93-c7b262605968.lock 2024-04-23T18:25:49.147 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying IP 172.21.15.132 port 3300 ... 2024-04-23T18:25:49.147 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying IP 172.21.15.132 port 6789 ... 2024-04-23T18:25:49.147 INFO:teuthology.orchestra.run.smithi132.stdout:Base mon IP(s) is [172.21.15.132:3300, 172.21.15.132:6789], mon addrv is [v2:172.21.15.132:3300,v1:172.21.15.132:6789] 2024-04-23T18:25:49.150 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.132 metric 100 2024-04-23T18:25:49.150 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.132 metric 100 2024-04-23T18:25:49.153 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-23T18:25:49.153 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-23T18:25:49.153 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-23T18:25:49.155 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-23T18:25:49.155 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-23T18:25:49.155 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T18:25:49.155 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-23T18:25:49.155 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout inet6 fe80::502d:ac94:fffc:d962/64 scope link noprefixroute 2024-04-23T18:25:49.155 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T18:25:49.156 INFO:teuthology.orchestra.run.smithi132.stdout:Mon IP `172.21.15.132` is in CIDR network `172.21.0.0/20` 2024-04-23T18:25:49.156 INFO:teuthology.orchestra.run.smithi132.stdout:Mon IP `172.21.15.132` is in CIDR network `172.21.0.0/20` 2024-04-23T18:25:49.156 INFO:teuthology.orchestra.run.smithi132.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-23T18:25:49.156 INFO:teuthology.orchestra.run.smithi132.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-23T18:25:49.157 INFO:teuthology.orchestra.run.smithi132.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stderr Getting image source signatures 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-23T18:25:53.602 INFO:teuthology.orchestra.run.smithi132.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:25:53.602 INFO:teuthology.orchestra.run.smithi132.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:25:53.602 INFO:teuthology.orchestra.run.smithi132.stdout:Extracting ceph user uid/gid from container image... 2024-04-23T18:25:54.610 INFO:teuthology.orchestra.run.smithi132.stdout:stat: stdout 167 167 2024-04-23T18:25:54.611 INFO:teuthology.orchestra.run.smithi132.stdout:Creating initial keys... 2024-04-23T18:25:55.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph-authtool: stdout AQAy/SdmPD/QNxAAGYkJJT3GqhB6BuWeMzHBNg== 2024-04-23T18:25:56.360 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph-authtool: stdout AQAz/SdmFEHxLhAA/0Nw2fFvbKgsC8PNkOWbLA== 2024-04-23T18:25:57.217 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph-authtool: stdout AQA0/SdmC5X3KhAAJGmJy65RcTZQL+J4gcof5g== 2024-04-23T18:25:57.218 INFO:teuthology.orchestra.run.smithi132.stdout:Creating initial monmap... 2024-04-23T18:25:58.424 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T18:25:58.424 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-23T18:25:58.424 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:25:58.424 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T18:25:58.424 INFO:teuthology.orchestra.run.smithi132.stdout:monmaptool for a [v2:172.21.15.132:3300,v1:172.21.15.132:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T18:25:58.424 INFO:teuthology.orchestra.run.smithi132.stdout:setting min_mon_release = quincy 2024-04-23T18:25:58.424 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/monmaptool: set fsid to c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:25:58.424 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T18:25:58.424 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:25:58.424 INFO:teuthology.orchestra.run.smithi132.stdout:Creating mon... 2024-04-23T18:25:59.346 INFO:teuthology.orchestra.run.smithi132.stdout:create mon.a on 2024-04-23T18:25:59.545 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-23T18:25:59.684 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-23T18:25:59.868 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c34642b4-019e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-c34642b4-019e-11ef-bc93-c7b262605968.target. 2024-04-23T18:25:59.868 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c34642b4-019e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-c34642b4-019e-11ef-bc93-c7b262605968.target. 2024-04-23T18:26:00.106 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.a 2024-04-23T18:26:00.106 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-23T18:26:00.278 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c34642b4-019e-11ef-bc93-c7b262605968.target.wants/ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c34642b4-019e-11ef-bc93-c7b262605968@.service. 2024-04-23T18:26:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:00 smithi132 podman[28770]: 2024-04-23 18:26:00.56975093 +0000 UTC m=+0.139715311 container create acafcffb14c9c97b4ebd2494388000b3672272b02b56e6cf3e47edcd8889e31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:26:00.819 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:26:00.819 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout disabled 2024-04-23T18:26:00.826 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:26:00.827 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout inactive 2024-04-23T18:26:00.827 INFO:teuthology.orchestra.run.smithi132.stdout:firewalld.service is not enabled 2024-04-23T18:26:00.827 INFO:teuthology.orchestra.run.smithi132.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T18:26:00.827 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for mon to start... 2024-04-23T18:26:00.827 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for mon... 2024-04-23T18:26:00.949 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:00 smithi132 podman[28770]: 2024-04-23 18:26:00.738720107 +0000 UTC m=+0.308684490 container init acafcffb14c9c97b4ebd2494388000b3672272b02b56e6cf3e47edcd8889e31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:26:00.949 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:00 smithi132 podman[28770]: 2024-04-23 18:26:00.741847319 +0000 UTC m=+0.311811701 container start acafcffb14c9c97b4ebd2494388000b3672272b02b56e6cf3e47edcd8889e31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-23T18:26:00.949 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:00 smithi132 ceph-mon[28798]: mkfs c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:26:00.949 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:00 smithi132 ceph-mon[28798]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T18:26:00.949 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:00 smithi132 bash[28770]: acafcffb14c9c97b4ebd2494388000b3672272b02b56e6cf3e47edcd8889e31b 2024-04-23T18:26:00.949 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:00 smithi132 systemd[1]: Started Ceph mon.a for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:26:01.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout cluster: 2024-04-23T18:26:01.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout id: c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:26:01.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-23T18:26:01.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:01.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout services: 2024-04-23T18:26:01.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.560185s) 2024-04-23T18:26:01.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-23T18:26:01.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-23T18:26:01.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:01.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout data: 2024-04-23T18:26:01.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout pgs: 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.264+0000 7f1107616700 1 Processor -- start 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.264+0000 7f1107616700 1 -- start start 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.265+0000 7f1107616700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 0x7f110007c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.265+0000 7f1107616700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f110007c910 con 0x7f110007de90 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.265+0000 7f11053b2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 0x7f110007c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.265+0000 7f11053b2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 0x7f110007c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51116/0 (socket says 172.21.15.132:51116) 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.265+0000 7f11053b2700 1 -- 172.21.15.132:0/1359789223 learned_addr learned my addr 172.21.15.132:0/1359789223 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.265+0000 7f11053b2700 1 -- 172.21.15.132:0/1359789223 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110007ca90 con 0x7f110007de90 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.265+0000 7f11053b2700 1 --2- 172.21.15.132:0/1359789223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 0x7f110007c340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0098a0 tx=0x7f10fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44beb3a7754bb22 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.266+0000 7f10f7fff700 1 -- 172.21.15.132:0/1359789223 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10fc004160 con 0x7f110007de90 2024-04-23T18:26:01.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.266+0000 7f10f7fff700 1 -- 172.21.15.132:0/1359789223 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10fc004300 con 0x7f110007de90 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.266+0000 7f10f7fff700 1 -- 172.21.15.132:0/1359789223 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10fc0036a0 con 0x7f110007de90 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.266+0000 7f1107616700 1 -- 172.21.15.132:0/1359789223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 msgr2=0x7f110007c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.266+0000 7f1107616700 1 --2- 172.21.15.132:0/1359789223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 0x7f110007c340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0098a0 tx=0x7f10fc009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.266+0000 7f1107616700 1 -- 172.21.15.132:0/1359789223 shutdown_connections 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.266+0000 7f1107616700 1 --2- 172.21.15.132:0/1359789223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 0x7f110007c340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.266+0000 7f1107616700 1 -- 172.21.15.132:0/1359789223 >> 172.21.15.132:0/1359789223 conn(0x7f1100103a80 msgr2=0x7f1100105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.267+0000 7f1107616700 1 -- 172.21.15.132:0/1359789223 shutdown_connections 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.267+0000 7f1107616700 1 -- 172.21.15.132:0/1359789223 wait complete. 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.267+0000 7f1107616700 1 Processor -- start 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.267+0000 7f1107616700 1 -- start start 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.268+0000 7f1107616700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 0x7f110019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.268+0000 7f1107616700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f110007cf60 con 0x7f110007de90 2024-04-23T18:26:01.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.268+0000 7f11053b2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 0x7f110019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:01.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.268+0000 7f11053b2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 0x7f110019dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51128/0 (socket says 172.21.15.132:51128) 2024-04-23T18:26:01.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.268+0000 7f11053b2700 1 -- 172.21.15.132:0/347464983 learned_addr learned my addr 172.21.15.132:0/347464983 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:01.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.268+0000 7f11053b2700 1 -- 172.21.15.132:0/347464983 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110019e240 con 0x7f110007de90 2024-04-23T18:26:01.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.268+0000 7f11053b2700 1 --2- 172.21.15.132:0/347464983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 0x7f110019dd00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f10fc000c00 tx=0x7f10fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:01.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.268+0000 7f10f67fc700 1 -- 172.21.15.132:0/347464983 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10fc003db0 con 0x7f110007de90 2024-04-23T18:26:01.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.268+0000 7f10f67fc700 1 -- 172.21.15.132:0/347464983 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10fc011630 con 0x7f110007de90 2024-04-23T18:26:01.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.268+0000 7f1107616700 1 -- 172.21.15.132:0/347464983 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110019e4d0 con 0x7f110007de90 2024-04-23T18:26:01.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.268+0000 7f10f67fc700 1 -- 172.21.15.132:0/347464983 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10fc011950 con 0x7f110007de90 2024-04-23T18:26:01.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.268+0000 7f1107616700 1 -- 172.21.15.132:0/347464983 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11001a11b0 con 0x7f110007de90 2024-04-23T18:26:01.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.269+0000 7f10f67fc700 1 -- 172.21.15.132:0/347464983 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f10fc01a460 con 0x7f110007de90 2024-04-23T18:26:01.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.269+0000 7f10f67fc700 1 -- 172.21.15.132:0/347464983 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f10fc01b780 con 0x7f110007de90 2024-04-23T18:26:01.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.269+0000 7f1107616700 1 -- 172.21.15.132:0/347464983 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11000640a0 con 0x7f110007de90 2024-04-23T18:26:01.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.272+0000 7f10f67fc700 1 -- 172.21.15.132:0/347464983 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f10fc031020 con 0x7f110007de90 2024-04-23T18:26:01.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.342+0000 7f1107616700 1 -- 172.21.15.132:0/347464983 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f110002e190 con 0x7f110007de90 2024-04-23T18:26:01.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.343+0000 7f10f67fc700 1 -- 172.21.15.132:0/347464983 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f10fc01b980 con 0x7f110007de90 2024-04-23T18:26:01.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.344+0000 7f1107616700 1 -- 172.21.15.132:0/347464983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 msgr2=0x7f110019dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:01.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.344+0000 7f1107616700 1 --2- 172.21.15.132:0/347464983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 0x7f110019dd00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f10fc000c00 tx=0x7f10fc004750 comp rx=0 tx=0).stop 2024-04-23T18:26:01.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.344+0000 7f1107616700 1 -- 172.21.15.132:0/347464983 shutdown_connections 2024-04-23T18:26:01.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.344+0000 7f1107616700 1 --2- 172.21.15.132:0/347464983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f110007de90 0x7f110019dd00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.344+0000 7f1107616700 1 -- 172.21.15.132:0/347464983 >> 172.21.15.132:0/347464983 conn(0x7f1100103a80 msgr2=0x7f11001048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:01.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.344+0000 7f1107616700 1 -- 172.21.15.132:0/347464983 shutdown_connections 2024-04-23T18:26:01.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:01.344+0000 7f1107616700 1 -- 172.21.15.132:0/347464983 wait complete. 2024-04-23T18:26:01.903 INFO:teuthology.orchestra.run.smithi132.stdout:mon is available 2024-04-23T18:26:01.903 INFO:teuthology.orchestra.run.smithi132.stdout:Assimilating anything we can from ceph.conf... 2024-04-23T18:26:02.021 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:01 smithi132 ceph-mon[28798]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T18:26:02.021 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:01 smithi132 ceph-mon[28798]: monmap epoch 1 2024-04-23T18:26:02.021 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:01 smithi132 ceph-mon[28798]: fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:26:02.021 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:01 smithi132 ceph-mon[28798]: last_changed 2024-04-23T18:25:57.540011+0000 2024-04-23T18:26:02.021 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:01 smithi132 ceph-mon[28798]: created 2024-04-23T18:25:57.540011+0000 2024-04-23T18:26:02.021 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:01 smithi132 ceph-mon[28798]: min_mon_release 19 (squid) 2024-04-23T18:26:02.021 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:01 smithi132 ceph-mon[28798]: election_strategy: 1 2024-04-23T18:26:02.021 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:01 smithi132 ceph-mon[28798]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-04-23T18:26:02.021 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:01 smithi132 ceph-mon[28798]: fsmap 2024-04-23T18:26:02.021 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:01 smithi132 ceph-mon[28798]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T18:26:02.021 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:01 smithi132 ceph-mon[28798]: mgrmap e1: no daemons active 2024-04-23T18:26:02.021 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:01 smithi132 ceph-mon[28798]: from='client.? 172.21.15.132:0/347464983' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-23T18:26:02.973 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:02.973 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T18:26:02.973 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout fsid = c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.132:3300,v1:172.21.15.132:6789] 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T18:26:02.974 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.365+0000 7f9cb100d700 1 Processor -- start 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.365+0000 7f9cb100d700 1 -- start start 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.366+0000 7f9cb100d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 0x7f9cac10b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.366+0000 7f9cb100d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac07be50 con 0x7f9cac108f50 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.366+0000 7f9caad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 0x7f9cac10b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.366+0000 7f9caad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 0x7f9cac10b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51144/0 (socket says 172.21.15.132:51144) 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.366+0000 7f9caad9d700 1 -- 172.21.15.132:0/915358325 learned_addr learned my addr 172.21.15.132:0/915358325 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.366+0000 7f9caad9d700 1 -- 172.21.15.132:0/915358325 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac10b880 con 0x7f9cac108f50 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.366+0000 7f9caad9d700 1 --2- 172.21.15.132:0/915358325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 0x7f9cac10b340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9c940098a0 tx=0x7f9c94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98a173dcbd34406 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:02.975 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.367+0000 7f9ca9d9b700 1 -- 172.21.15.132:0/915358325 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c94004160 con 0x7f9cac108f50 2024-04-23T18:26:02.976 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.367+0000 7f9ca9d9b700 1 -- 172.21.15.132:0/915358325 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c94004300 con 0x7f9cac108f50 2024-04-23T18:26:02.976 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.367+0000 7f9ca9d9b700 1 -- 172.21.15.132:0/915358325 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c940036a0 con 0x7f9cac108f50 2024-04-23T18:26:02.976 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.367+0000 7f9cb100d700 1 -- 172.21.15.132:0/915358325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 msgr2=0x7f9cac10b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:02.976 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.367+0000 7f9cb100d700 1 --2- 172.21.15.132:0/915358325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 0x7f9cac10b340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9c940098a0 tx=0x7f9c94009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:02.976 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.367+0000 7f9cb100d700 1 -- 172.21.15.132:0/915358325 shutdown_connections 2024-04-23T18:26:02.976 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.367+0000 7f9cb100d700 1 --2- 172.21.15.132:0/915358325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 0x7f9cac10b340 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:02.976 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.367+0000 7f9cb100d700 1 -- 172.21.15.132:0/915358325 >> 172.21.15.132:0/915358325 conn(0x7f9cac102940 msgr2=0x7f9cac104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:02.976 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.367+0000 7f9cb100d700 1 -- 172.21.15.132:0/915358325 shutdown_connections 2024-04-23T18:26:02.976 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.367+0000 7f9cb100d700 1 -- 172.21.15.132:0/915358325 wait complete. 2024-04-23T18:26:02.976 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.368+0000 7f9cb100d700 1 Processor -- start 2024-04-23T18:26:02.976 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.368+0000 7f9cb100d700 1 -- start start 2024-04-23T18:26:02.976 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.368+0000 7f9cb100d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 0x7f9cac19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.368+0000 7f9cb100d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac10bcf0 con 0x7f9cac108f50 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.368+0000 7f9caad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 0x7f9cac19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.369+0000 7f9caad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 0x7f9cac19bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51146/0 (socket says 172.21.15.132:51146) 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.369+0000 7f9caad9d700 1 -- 172.21.15.132:0/1613728970 learned_addr learned my addr 172.21.15.132:0/1613728970 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.369+0000 7f9caad9d700 1 -- 172.21.15.132:0/1613728970 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac19c0b0 con 0x7f9cac108f50 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.369+0000 7f9caad9d700 1 --2- 172.21.15.132:0/1613728970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 0x7f9cac19bb70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9c94000c00 tx=0x7f9c94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.369+0000 7f9ca3fff700 1 -- 172.21.15.132:0/1613728970 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c94003db0 con 0x7f9cac108f50 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.369+0000 7f9ca3fff700 1 -- 172.21.15.132:0/1613728970 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c94011630 con 0x7f9cac108f50 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.369+0000 7f9cb100d700 1 -- 172.21.15.132:0/1613728970 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cac19c340 con 0x7f9cac108f50 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.369+0000 7f9ca3fff700 1 -- 172.21.15.132:0/1613728970 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c94011950 con 0x7f9cac108f50 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.369+0000 7f9cb100d700 1 -- 172.21.15.132:0/1613728970 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cac19f030 con 0x7f9cac108f50 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.369+0000 7f9ca3fff700 1 -- 172.21.15.132:0/1613728970 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9c9401a460 con 0x7f9cac108f50 2024-04-23T18:26:02.977 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.369+0000 7f9ca3fff700 1 -- 172.21.15.132:0/1613728970 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9c9401b780 con 0x7f9cac108f50 2024-04-23T18:26:02.978 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.370+0000 7f9cb100d700 1 -- 172.21.15.132:0/1613728970 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cac0640a0 con 0x7f9cac108f50 2024-04-23T18:26:02.978 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.372+0000 7f9ca3fff700 1 -- 172.21.15.132:0/1613728970 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9c94031020 con 0x7f9cac108f50 2024-04-23T18:26:02.978 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.439+0000 7f9cb100d700 1 -- 172.21.15.132:0/1613728970 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9cac02e280 con 0x7f9cac108f50 2024-04-23T18:26:02.978 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.442+0000 7f9ca3fff700 1 -- 172.21.15.132:0/1613728970 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+467 (secure 0 0 0) 0x7f9c9401b9d0 con 0x7f9cac108f50 2024-04-23T18:26:02.978 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.443+0000 7f9ca3fff700 1 -- 172.21.15.132:0/1613728970 <== mon.0 v2:172.21.15.132:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9c9401ad20 con 0x7f9cac108f50 2024-04-23T18:26:02.978 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.444+0000 7f9cb100d700 1 -- 172.21.15.132:0/1613728970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 msgr2=0x7f9cac19bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:02.978 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.444+0000 7f9cb100d700 1 --2- 172.21.15.132:0/1613728970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 0x7f9cac19bb70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9c94000c00 tx=0x7f9c94004750 comp rx=0 tx=0).stop 2024-04-23T18:26:02.978 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.444+0000 7f9cb100d700 1 -- 172.21.15.132:0/1613728970 shutdown_connections 2024-04-23T18:26:02.978 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.444+0000 7f9cb100d700 1 --2- 172.21.15.132:0/1613728970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cac108f50 0x7f9cac19bb70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:02.979 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.444+0000 7f9cb100d700 1 -- 172.21.15.132:0/1613728970 >> 172.21.15.132:0/1613728970 conn(0x7f9cac102940 msgr2=0x7f9cac104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:02.979 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.444+0000 7f9cb100d700 1 -- 172.21.15.132:0/1613728970 shutdown_connections 2024-04-23T18:26:02.979 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:02.444+0000 7f9cb100d700 1 -- 172.21.15.132:0/1613728970 wait complete. 2024-04-23T18:26:02.979 INFO:teuthology.orchestra.run.smithi132.stdout:Generating new minimal ceph.conf... 2024-04-23T18:26:03.089 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:02 smithi132 ceph-mon[28798]: from='client.? 172.21.15.132:0/1613728970' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T18:26:03.090 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:02 smithi132 ceph-mon[28798]: from='client.? 172.21.15.132:0/1613728970' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-23T18:26:03.930 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.371+0000 7ff5f65f0700 1 Processor -- start 2024-04-23T18:26:03.930 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.371+0000 7ff5f65f0700 1 -- start start 2024-04-23T18:26:03.930 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.371+0000 7ff5f65f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 0x7ff5f0107710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:03.930 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.371+0000 7ff5f65f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f0107ce0 con 0x7ff5f01072f0 2024-04-23T18:26:03.930 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.372+0000 7ff5effff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 0x7ff5f0107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:03.930 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.372+0000 7ff5effff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 0x7ff5f0107710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51154/0 (socket says 172.21.15.132:51154) 2024-04-23T18:26:03.930 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.372+0000 7ff5effff700 1 -- 172.21.15.132:0/3116502030 learned_addr learned my addr 172.21.15.132:0/3116502030 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:03.930 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.372+0000 7ff5effff700 1 -- 172.21.15.132:0/3116502030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f01084f0 con 0x7ff5f01072f0 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.372+0000 7ff5effff700 1 --2- 172.21.15.132:0/3116502030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 0x7ff5f0107710 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40098a0 tx=0x7ff5e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41c2e5e1be2dad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.372+0000 7ff5eeffd700 1 -- 172.21.15.132:0/3116502030 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e4004160 con 0x7ff5f01072f0 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.372+0000 7ff5eeffd700 1 -- 172.21.15.132:0/3116502030 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff5e4004300 con 0x7ff5f01072f0 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.372+0000 7ff5eeffd700 1 -- 172.21.15.132:0/3116502030 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e4011af0 con 0x7ff5f01072f0 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.373+0000 7ff5f65f0700 1 -- 172.21.15.132:0/3116502030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 msgr2=0x7ff5f0107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.373+0000 7ff5f65f0700 1 --2- 172.21.15.132:0/3116502030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 0x7ff5f0107710 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40098a0 tx=0x7ff5e4009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.373+0000 7ff5f65f0700 1 -- 172.21.15.132:0/3116502030 shutdown_connections 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.373+0000 7ff5f65f0700 1 --2- 172.21.15.132:0/3116502030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 0x7ff5f0107710 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.373+0000 7ff5f65f0700 1 -- 172.21.15.132:0/3116502030 >> 172.21.15.132:0/3116502030 conn(0x7ff5f0102890 msgr2=0x7ff5f0104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.373+0000 7ff5f65f0700 1 -- 172.21.15.132:0/3116502030 shutdown_connections 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.373+0000 7ff5f65f0700 1 -- 172.21.15.132:0/3116502030 wait complete. 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.373+0000 7ff5f65f0700 1 Processor -- start 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.373+0000 7ff5f65f0700 1 -- start start 2024-04-23T18:26:03.931 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.374+0000 7ff5f65f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 0x7ff5f019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.374+0000 7ff5f65f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f010b8a0 con 0x7ff5f01072f0 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.374+0000 7ff5effff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 0x7ff5f019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.374+0000 7ff5effff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 0x7ff5f019c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51164/0 (socket says 172.21.15.132:51164) 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.374+0000 7ff5effff700 1 -- 172.21.15.132:0/2647437480 learned_addr learned my addr 172.21.15.132:0/2647437480 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.374+0000 7ff5effff700 1 -- 172.21.15.132:0/2647437480 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f019c960 con 0x7ff5f01072f0 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.374+0000 7ff5effff700 1 --2- 172.21.15.132:0/2647437480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 0x7ff5f019c420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40117b0 tx=0x7ff5e4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.374+0000 7ff5ed7fa700 1 -- 172.21.15.132:0/2647437480 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e4003af0 con 0x7ff5f01072f0 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.374+0000 7ff5f65f0700 1 -- 172.21.15.132:0/2647437480 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f019cbf0 con 0x7ff5f01072f0 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.374+0000 7ff5ed7fa700 1 -- 172.21.15.132:0/2647437480 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff5e4020070 con 0x7ff5f01072f0 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.375+0000 7ff5ed7fa700 1 -- 172.21.15.132:0/2647437480 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e4019630 con 0x7ff5f01072f0 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.375+0000 7ff5f65f0700 1 -- 172.21.15.132:0/2647437480 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f019f8e0 con 0x7ff5f01072f0 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.375+0000 7ff5ed7fa700 1 -- 172.21.15.132:0/2647437480 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff5e4019c80 con 0x7ff5f01072f0 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.375+0000 7ff5ed7fa700 1 -- 172.21.15.132:0/2647437480 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff5e400b720 con 0x7ff5f01072f0 2024-04-23T18:26:03.932 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.376+0000 7ff5f65f0700 1 -- 172.21.15.132:0/2647437480 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f00640a0 con 0x7ff5f01072f0 2024-04-23T18:26:03.933 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.378+0000 7ff5ed7fa700 1 -- 172.21.15.132:0/2647437480 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff5e40042a0 con 0x7ff5f01072f0 2024-04-23T18:26:03.933 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.445+0000 7ff5f65f0700 1 -- 172.21.15.132:0/2647437480 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff5f002e190 con 0x7ff5f01072f0 2024-04-23T18:26:03.933 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.446+0000 7ff5ed7fa700 1 -- 172.21.15.132:0/2647437480 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7ff5e401a9c0 con 0x7ff5f01072f0 2024-04-23T18:26:03.933 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.446+0000 7ff5f65f0700 1 -- 172.21.15.132:0/2647437480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 msgr2=0x7ff5f019c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:03.933 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.446+0000 7ff5f65f0700 1 --2- 172.21.15.132:0/2647437480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 0x7ff5f019c420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40117b0 tx=0x7ff5e4011b70 comp rx=0 tx=0).stop 2024-04-23T18:26:03.933 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.446+0000 7ff5f65f0700 1 -- 172.21.15.132:0/2647437480 shutdown_connections 2024-04-23T18:26:03.933 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.446+0000 7ff5f65f0700 1 --2- 172.21.15.132:0/2647437480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5f01072f0 0x7ff5f019c420 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:03.933 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.446+0000 7ff5f65f0700 1 -- 172.21.15.132:0/2647437480 >> 172.21.15.132:0/2647437480 conn(0x7ff5f0102890 msgr2=0x7ff5f01037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:03.933 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.447+0000 7ff5f65f0700 1 -- 172.21.15.132:0/2647437480 shutdown_connections 2024-04-23T18:26:03.933 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:03.447+0000 7ff5f65f0700 1 -- 172.21.15.132:0/2647437480 wait complete. 2024-04-23T18:26:03.933 INFO:teuthology.orchestra.run.smithi132.stdout:Restarting the monitor... 2024-04-23T18:26:04.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:03 smithi132 ceph-mon[28798]: from='client.? 172.21.15.132:0/2647437480' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:04.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:03 smithi132 systemd[1]: Stopping Ceph mon.a for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:26:04.476 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:04 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a[28794]: 2024-04-23T18:26:04.199+0000 7f2c9763b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:26:04.476 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:04 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a[28794]: 2024-04-23T18:26:04.199+0000 7f2c9763b700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-23T18:26:04.476 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:04 smithi132 podman[29044]: 2024-04-23 18:26:04.221111329 +0000 UTC m=+0.114569848 container died acafcffb14c9c97b4ebd2494388000b3672272b02b56e6cf3e47edcd8889e31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-04-23T18:26:04.835 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:04 smithi132 podman[29044]: 2024-04-23 18:26:04.583605397 +0000 UTC m=+0.477063957 container cleanup acafcffb14c9c97b4ebd2494388000b3672272b02b56e6cf3e47edcd8889e31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-23T18:26:04.835 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:04 smithi132 bash[29044]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a 2024-04-23T18:26:05.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:04 smithi132 podman[29057]: 2024-04-23 18:26:04.843004015 +0000 UTC m=+0.622718307 container remove acafcffb14c9c97b4ebd2494388000b3672272b02b56e6cf3e47edcd8889e31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-23T18:26:05.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 systemd[1]: ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-23T18:26:05.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 systemd[1]: Stopped Ceph mon.a for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:26:05.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 systemd[1]: Starting Ceph mon.a for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:26:05.460 INFO:teuthology.orchestra.run.smithi132.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-23T18:26:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 podman[29159]: 2024-04-23 18:26:05.218910566 +0000 UTC m=+0.114700817 container create 2bb1a8944af28005f4862de28709bdc377209a0bf3acbc624102a9c218734d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-23T18:26:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 podman[29159]: 2024-04-23 18:26:05.12214614 +0000 UTC m=+0.017936369 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:26:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 podman[29159]: 2024-04-23 18:26:05.388216059 +0000 UTC m=+0.284006261 container init 2bb1a8944af28005f4862de28709bdc377209a0bf3acbc624102a9c218734d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-23T18:26:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 podman[29159]: 2024-04-23 18:26:05.391157807 +0000 UTC m=+0.286948010 container start 2bb1a8944af28005f4862de28709bdc377209a0bf3acbc624102a9c218734d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-23T18:26:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T18:26:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-23T18:26:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: pidfile_write: ignore empty --pid-file 2024-04-23T18:26:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: load: jerasure load: lrc 2024-04-23T18:26:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: RocksDB version: 7.9.2 2024-04-23T18:26:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Git sha 0 2024-04-23T18:26:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-23T18:26:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: DB SUMMARY 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: DB Session ID: Q4QNPHEYQK14ERE1XRFY 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: CURRENT file: CURRENT 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105148 ; 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.error_if_exists: 0 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.create_if_missing: 0 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.paranoid_checks: 1 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T18:26:05.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.env: 0x5627d9cc2cc0 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.info_log: 0x5627daf2fbe0 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.statistics: (nil) 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.use_fsync: 0 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_log_file_size: 0 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.allow_fallocate: 1 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.use_direct_reads: 0 2024-04-23T18:26:05.480 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.db_log_dir: 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.wal_dir: 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.write_buffer_manager: 0x5627daec30e0 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T18:26:05.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T18:26:05.482 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T18:26:05.482 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T18:26:05.482 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T18:26:05.483 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.unordered_write: 0 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.row_cache: None 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.wal_filter: None 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.two_write_queues: 0 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T18:26:05.484 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.wal_compression: 0 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.atomic_flush: 0 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.log_readahead_size: 0 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T18:26:05.485 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_background_jobs: 2 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_background_compactions: -1 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_subcompactions: 1 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_open_files: -1 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T18:26:05.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_background_flushes: -1 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Compression algorithms supported: 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: kZSTD supported: 0 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: kXpressCompression supported: 0 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: kZlibCompression supported: 1 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: kSnappyCompression supported: 1 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: kLZ4Compression supported: 1 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: kBZip2Compression supported: 0 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-23T18:26:05.487 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.merge_operator: 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_filter: None 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5627daf2fdc0) 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout: cache_index_and_filter_blocks: 1 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout: pin_top_level_index_and_filter: 1 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout: index_type: 0 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout: data_block_index_type: 0 2024-04-23T18:26:05.488 INFO:journalctl@ceph.mon.a.smithi132.stdout: index_shortening: 1 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: checksum: 4 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: no_block_cache: 0 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_cache: 0x5627daef51f0 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_cache_name: BinnedLRUCache 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_cache_options: 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: capacity : 536870912 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: num_shard_bits : 4 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: strict_capacity_limit : 0 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: high_pri_pool_ratio: 0.000 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_cache_compressed: (nil) 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: persistent_cache: (nil) 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_size: 4096 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_size_deviation: 10 2024-04-23T18:26:05.489 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_restart_interval: 16 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: index_block_restart_interval: 1 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: metadata_block_size: 4096 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: partition_filters: 0 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: use_delta_encoding: 1 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: filter_policy: bloomfilter 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: whole_key_filtering: 1 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: verify_compression: 0 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: read_amp_bytes_per_bit: 0 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: format_version: 5 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: enable_index_compression: 1 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_align: 0 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: max_auto_readahead_size: 262144 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: prepopulate_block_cache: 0 2024-04-23T18:26:05.490 INFO:journalctl@ceph.mon.a.smithi132.stdout: initial_auto_readahead_size: 8192 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compression: NoCompression 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.num_levels: 7 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T18:26:05.491 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T18:26:05.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:26:05.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T18:26:05.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T18:26:05.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T18:26:05.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T18:26:05.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T18:26:05.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T18:26:05.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T18:26:05.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T18:26:05.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T18:26:05.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T18:26:05.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T18:26:05.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.inplace_update_support: 0 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.bloom_locality: 0 2024-04-23T18:26:05.495 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.max_successive_merges: 0 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.ttl: 2592000 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.enable_blob_files: false 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.min_blob_size: 0 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T18:26:05.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 932ab89b-ac5d-431f-af5a-5b4ba546f329 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896765419357, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896765420383, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100315, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98200, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14169, "raw_average_key_size": 49, "raw_value_size": 90484, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713896765, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "932ab89b-ac5d-431f-af5a-5b4ba546f329", "db_session_id": "Q4QNPHEYQK14ERE1XRFY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896765420466, "job": 1, "event": "recovery_finished"} 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5627db00a000 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: DB pointer 0x5627daff4000 2024-04-23T18:26:05.497 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: ** DB Stats ** 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: ** Compaction Stats [default] ** 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T18:26:05.498 INFO:journalctl@ceph.mon.a.smithi132.stdout: L0 2/0 99.82 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: Sum 2/0 99.82 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: ** Compaction Stats [default] ** 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 145.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T18:26:05.499 INFO:journalctl@ceph.mon.a.smithi132.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout: Cumulative compaction: 0.00 GB write, 31.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout: Interval compaction: 0.00 GB write, 31.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout: Block cache BinnedLRUCache@0x5627daef51f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout: 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: starting mon.a rank 0 at public addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] at bind addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: mon.a@-1(???) e1 preinit fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: mon.a@-1(???).mds e1 new map 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: mon.a@-1(???).mds e1 print_map 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout: e1 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T18:26:05.500 INFO:journalctl@ceph.mon.a.smithi132.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout: legacy client fscid: -1 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout: 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout: No filesystems configured 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 bash[29159]: 2bb1a8944af28005f4862de28709bdc377209a0bf3acbc624102a9c218734d7f 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 systemd[1]: Started Ceph mon.a for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: monmap epoch 1 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: last_changed 2024-04-23T18:25:57.540011+0000 2024-04-23T18:26:05.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: created 2024-04-23T18:25:57.540011+0000 2024-04-23T18:26:05.502 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: min_mon_release 19 (squid) 2024-04-23T18:26:05.502 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: election_strategy: 1 2024-04-23T18:26:05.502 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-04-23T18:26:05.502 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: fsmap 2024-04-23T18:26:05.502 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T18:26:05.502 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:05 smithi132 ceph-mon[29178]: mgrmap e1: no daemons active 2024-04-23T18:26:06.535 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.867+0000 7f6ce4e45700 1 Processor -- start 2024-04-23T18:26:06.535 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.867+0000 7f6ce4e45700 1 -- start start 2024-04-23T18:26:06.535 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.867+0000 7f6ce4e45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 0x7f6ce007c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:06.535 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.867+0000 7f6ce4e45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce007c910 con 0x7f6ce007de90 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.867+0000 7f6cde59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 0x7f6ce007c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.867+0000 7f6cde59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 0x7f6ce007c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51166/0 (socket says 172.21.15.132:51166) 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.867+0000 7f6cde59c700 1 -- 172.21.15.132:0/1699899415 learned_addr learned my addr 172.21.15.132:0/1699899415 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.868+0000 7f6cde59c700 1 -- 172.21.15.132:0/1699899415 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce007ca90 con 0x7f6ce007de90 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.868+0000 7f6cde59c700 1 --2- 172.21.15.132:0/1699899415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 0x7f6ce007c340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00098a0 tx=0x7f6cd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71305d0a77d70fd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.868+0000 7f6cdd59a700 1 -- 172.21.15.132:0/1699899415 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cd0004160 con 0x7f6ce007de90 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.868+0000 7f6cdd59a700 1 -- 172.21.15.132:0/1699899415 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6cd0004300 con 0x7f6ce007de90 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.868+0000 7f6cdd59a700 1 -- 172.21.15.132:0/1699899415 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cd0011af0 con 0x7f6ce007de90 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.869+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1699899415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 msgr2=0x7f6ce007c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.869+0000 7f6ce4e45700 1 --2- 172.21.15.132:0/1699899415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 0x7f6ce007c340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00098a0 tx=0x7f6cd0009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.869+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1699899415 shutdown_connections 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.869+0000 7f6ce4e45700 1 --2- 172.21.15.132:0/1699899415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 0x7f6ce007c340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.869+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1699899415 >> 172.21.15.132:0/1699899415 conn(0x7f6ce0103a80 msgr2=0x7f6ce0105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.869+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1699899415 shutdown_connections 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.869+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1699899415 wait complete. 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6ce4e45700 1 Processor -- start 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6ce4e45700 1 -- start start 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6ce4e45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 0x7f6ce019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6ce4e45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce010e990 con 0x7f6ce007de90 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6cde59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 0x7f6ce019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6cde59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 0x7f6ce019e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51168/0 (socket says 172.21.15.132:51168) 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6cde59c700 1 -- 172.21.15.132:0/1940052913 learned_addr learned my addr 172.21.15.132:0/1940052913 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6cde59c700 1 -- 172.21.15.132:0/1940052913 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce019eab0 con 0x7f6ce007de90 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6cde59c700 1 --2- 172.21.15.132:0/1940052913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 0x7f6ce019e570 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009fb0 tx=0x7f6cd0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6ccf7fe700 1 -- 172.21.15.132:0/1940052913 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cd00036a0 con 0x7f6ce007de90 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1940052913 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce019ed40 con 0x7f6ce007de90 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6ccf7fe700 1 -- 172.21.15.132:0/1940052913 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6cd001a430 con 0x7f6ce007de90 2024-04-23T18:26:06.537 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.870+0000 7f6ccf7fe700 1 -- 172.21.15.132:0/1940052913 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cd0019550 con 0x7f6ce007de90 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.871+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1940052913 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce01a1a30 con 0x7f6ce007de90 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.871+0000 7f6ccf7fe700 1 -- 172.21.15.132:0/1940052913 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6cd0019be0 con 0x7f6ce007de90 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.871+0000 7f6ccf7fe700 1 -- 172.21.15.132:0/1940052913 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6cd0024730 con 0x7f6ce007de90 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.871+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1940052913 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce00640a0 con 0x7f6ce007de90 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.874+0000 7f6ccf7fe700 1 -- 172.21.15.132:0/1940052913 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6cd0043e50 con 0x7f6ce007de90 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.942+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1940052913 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f6ce01a1c00 con 0x7f6ce007de90 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.944+0000 7f6ccf7fe700 1 -- 172.21.15.132:0/1940052913 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f6cd0032020 con 0x7f6ce007de90 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.945+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1940052913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 msgr2=0x7f6ce019e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.945+0000 7f6ce4e45700 1 --2- 172.21.15.132:0/1940052913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 0x7f6ce019e570 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009fb0 tx=0x7f6cd0011940 comp rx=0 tx=0).stop 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.945+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1940052913 shutdown_connections 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.945+0000 7f6ce4e45700 1 --2- 172.21.15.132:0/1940052913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce007de90 0x7f6ce019e570 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.945+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1940052913 >> 172.21.15.132:0/1940052913 conn(0x7f6ce0103a80 msgr2=0x7f6ce01045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:06.538 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.945+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1940052913 shutdown_connections 2024-04-23T18:26:06.539 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:05.945+0000 7f6ce4e45700 1 -- 172.21.15.132:0/1940052913 wait complete. 2024-04-23T18:26:06.539 INFO:teuthology.orchestra.run.smithi132.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-23T18:26:06.561 INFO:teuthology.orchestra.run.smithi132.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:06.561 INFO:teuthology.orchestra.run.smithi132.stdout:Creating mgr... 2024-04-23T18:26:06.562 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-23T18:26:06.562 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-23T18:26:06.795 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.x 2024-04-23T18:26:06.795 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-23T18:26:06.945 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c34642b4-019e-11ef-bc93-c7b262605968.target.wants/ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c34642b4-019e-11ef-bc93-c7b262605968@.service. 2024-04-23T18:26:07.188 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:06 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1940052913' entity='client.admin' 2024-04-23T18:26:07.189 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:07 smithi132 podman[29397]: 2024-04-23 18:26:07.093807737 +0000 UTC m=+0.017738949 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:26:07.189 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:07 smithi132 podman[29397]: 2024-04-23 18:26:07.196458737 +0000 UTC m=+0.120389982 container create 390d7bfdf89bc8ba87bcf5b271bf04255ee6b1ae71bf854b5e1e06d58485be30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-23T18:26:07.490 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:07 smithi132 podman[29397]: 2024-04-23 18:26:07.423690414 +0000 UTC m=+0.347621616 container init 390d7bfdf89bc8ba87bcf5b271bf04255ee6b1ae71bf854b5e1e06d58485be30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-23T18:26:07.491 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:07 smithi132 podman[29397]: 2024-04-23 18:26:07.426178898 +0000 UTC m=+0.350110100 container start 390d7bfdf89bc8ba87bcf5b271bf04255ee6b1ae71bf854b5e1e06d58485be30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:26:07.504 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:26:07.504 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout disabled 2024-04-23T18:26:07.511 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:26:07.511 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout inactive 2024-04-23T18:26:07.511 INFO:teuthology.orchestra.run.smithi132.stdout:firewalld.service is not enabled 2024-04-23T18:26:07.512 INFO:teuthology.orchestra.run.smithi132.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T18:26:07.519 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:26:07.520 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout disabled 2024-04-23T18:26:07.527 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:26:07.527 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout inactive 2024-04-23T18:26:07.527 INFO:teuthology.orchestra.run.smithi132.stdout:firewalld.service is not enabled 2024-04-23T18:26:07.527 INFO:teuthology.orchestra.run.smithi132.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-23T18:26:07.527 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for mgr to start... 2024-04-23T18:26:07.527 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for mgr... 2024-04-23T18:26:07.758 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:07 smithi132 bash[29397]: 390d7bfdf89bc8ba87bcf5b271bf04255ee6b1ae71bf854b5e1e06d58485be30 2024-04-23T18:26:07.758 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:07 smithi132 systemd[1]: Started Ceph mgr.x for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:26:07.758 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:07 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:07.580+0000 7f1bca38a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:26:07.758 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:07 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:07.764+0000 7f1bca38a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:08.009 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:07 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:07.871+0000 7f1bca38a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:26:08.273 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:08 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3994983329' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:26:08.273 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:08 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:08.195+0000 7f1bca38a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:26:08.273 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:08 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:08.280+0000 7f1bca38a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:26:08.635 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:26:08.636 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:26:08.637 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:26:08.637 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-23T18:26:08.637 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:26:08.637 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:08.637 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:26:08.637 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:26:08.637 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:08.637 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:26:08.637 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:08.637 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:26:08.637 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:26:08.637 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:26:08.645 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:26:08.645 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:26:08.645 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:26:08.645 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:08.645 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:26:08.645 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:26:08.645 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:26:08.645 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:26:08.645 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:26:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:26:00.784990+0000", 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.009+0000 7f9b0222a700 1 Processor -- start 2024-04-23T18:26:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.009+0000 7f9b0222a700 1 -- start start 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.009+0000 7f9b0222a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 0x7f9afc10b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.009+0000 7f9b0222a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc07be70 con 0x7f9afc108ef0 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.009+0000 7f9afb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 0x7f9afc10b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.009+0000 7f9afb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 0x7f9afc10b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51188/0 (socket says 172.21.15.132:51188) 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.009+0000 7f9afb7fe700 1 -- 172.21.15.132:0/2509154186 learned_addr learned my addr 172.21.15.132:0/2509154186 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.010+0000 7f9afb7fe700 1 -- 172.21.15.132:0/2509154186 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9afc10b820 con 0x7f9afc108ef0 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.010+0000 7f9afb7fe700 1 --2- 172.21.15.132:0/2509154186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 0x7f9afc10b2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9af00098a0 tx=0x7f9af0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a05296c69b24445 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.010+0000 7f9afa7fc700 1 -- 172.21.15.132:0/2509154186 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9af0004160 con 0x7f9afc108ef0 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.010+0000 7f9afa7fc700 1 -- 172.21.15.132:0/2509154186 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9af0004300 con 0x7f9afc108ef0 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.010+0000 7f9afa7fc700 1 -- 172.21.15.132:0/2509154186 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9af0011af0 con 0x7f9afc108ef0 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.011+0000 7f9b0222a700 1 -- 172.21.15.132:0/2509154186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 msgr2=0x7f9afc10b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.011+0000 7f9b0222a700 1 --2- 172.21.15.132:0/2509154186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 0x7f9afc10b2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9af00098a0 tx=0x7f9af0009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.011+0000 7f9b0222a700 1 -- 172.21.15.132:0/2509154186 shutdown_connections 2024-04-23T18:26:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.011+0000 7f9b0222a700 1 --2- 172.21.15.132:0/2509154186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 0x7f9afc10b2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.011+0000 7f9b0222a700 1 -- 172.21.15.132:0/2509154186 >> 172.21.15.132:0/2509154186 conn(0x7f9afc1028c0 msgr2=0x7f9afc104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.011+0000 7f9b0222a700 1 -- 172.21.15.132:0/2509154186 shutdown_connections 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.011+0000 7f9b0222a700 1 -- 172.21.15.132:0/2509154186 wait complete. 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.011+0000 7f9b0222a700 1 Processor -- start 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.012+0000 7f9b0222a700 1 -- start start 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.012+0000 7f9b0222a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 0x7f9afc108770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.012+0000 7f9b0222a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc10c790 con 0x7f9afc108ef0 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.012+0000 7f9afb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 0x7f9afc108770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.012+0000 7f9afb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 0x7f9afc108770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51202/0 (socket says 172.21.15.132:51202) 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.012+0000 7f9afb7fe700 1 -- 172.21.15.132:0/3994983329 learned_addr learned my addr 172.21.15.132:0/3994983329 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.012+0000 7f9afb7fe700 1 -- 172.21.15.132:0/3994983329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9afc108cb0 con 0x7f9afc108ef0 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.012+0000 7f9afb7fe700 1 --2- 172.21.15.132:0/3994983329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 0x7f9afc108770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009fb0 tx=0x7f9af0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.012+0000 7f9af8ff9700 1 -- 172.21.15.132:0/3994983329 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9af00036a0 con 0x7f9afc108ef0 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.013+0000 7f9af8ff9700 1 -- 172.21.15.132:0/3994983329 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9af001a430 con 0x7f9afc108ef0 2024-04-23T18:26:08.649 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.013+0000 7f9b0222a700 1 -- 172.21.15.132:0/3994983329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9afc106ea0 con 0x7f9afc108ef0 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.013+0000 7f9af8ff9700 1 -- 172.21.15.132:0/3994983329 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9af0019550 con 0x7f9afc108ef0 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.013+0000 7f9b0222a700 1 -- 172.21.15.132:0/3994983329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9afc1073e0 con 0x7f9afc108ef0 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.013+0000 7f9af8ff9700 1 -- 172.21.15.132:0/3994983329 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9af0019b90 con 0x7f9afc108ef0 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.013+0000 7f9af8ff9700 1 -- 172.21.15.132:0/3994983329 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9af0024730 con 0x7f9afc108ef0 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.013+0000 7f9b0222a700 1 -- 172.21.15.132:0/3994983329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9afc0640a0 con 0x7f9afc108ef0 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.016+0000 7f9af8ff9700 1 -- 172.21.15.132:0/3994983329 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9af001a5e0 con 0x7f9afc108ef0 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.089+0000 7f9b0222a700 1 -- 172.21.15.132:0/3994983329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9afc107690 con 0x7f9afc108ef0 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.090+0000 7f9af8ff9700 1 -- 172.21.15.132:0/3994983329 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9af001a8e0 con 0x7f9afc108ef0 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.091+0000 7f9b0222a700 1 -- 172.21.15.132:0/3994983329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 msgr2=0x7f9afc108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.091+0000 7f9b0222a700 1 --2- 172.21.15.132:0/3994983329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 0x7f9afc108770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009fb0 tx=0x7f9af0011940 comp rx=0 tx=0).stop 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.091+0000 7f9b0222a700 1 -- 172.21.15.132:0/3994983329 shutdown_connections 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.091+0000 7f9b0222a700 1 --2- 172.21.15.132:0/3994983329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9afc108ef0 0x7f9afc108770 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.091+0000 7f9b0222a700 1 -- 172.21.15.132:0/3994983329 >> 172.21.15.132:0/3994983329 conn(0x7f9afc1028c0 msgr2=0x7f9afc103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:08.650 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.091+0000 7f9b0222a700 1 -- 172.21.15.132:0/3994983329 shutdown_connections 2024-04-23T18:26:08.651 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:08.091+0000 7f9b0222a700 1 -- 172.21.15.132:0/3994983329 wait complete. 2024-04-23T18:26:08.651 INFO:teuthology.orchestra.run.smithi132.stdout:mgr not available, waiting (1/15)... 2024-04-23T18:26:09.818 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:09 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:09.487+0000 7f1bca38a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:26:09.818 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:09 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:09.654+0000 7f1bca38a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:09.818 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:09 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:09.825+0000 7f1bca38a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:26:10.362 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:10 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:10.108+0000 7f1bca38a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:26:10.362 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:10 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:10.199+0000 7f1bca38a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:26:10.764 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:10 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:10.404+0000 7f1bca38a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:26:10.764 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:10 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:10.593+0000 7f1bca38a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:11.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:11 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2865785030' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:26:11.362 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:11 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:11.223+0000 7f1bca38a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:26:11.720 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:11 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:11.650+0000 7f1bca38a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:26:11.726 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:11.726 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:26:11.726 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:26:11.726 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:26:11.726 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:26:11.726 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:26:11.729 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:26:11.729 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:11.729 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:26:11.729 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:11.730 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:26:11.731 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:26:11.731 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:26:11.738 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:26:11.738 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:26:11.738 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:26:11.738 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:11.738 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:11.739 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:26:00.784990+0000", 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:11.740 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.091+0000 7f5a99c68700 1 Processor -- start 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.091+0000 7f5a99c68700 1 -- start start 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.091+0000 7f5a99c68700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 0x7f5a9410b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.091+0000 7f5a99c68700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9407be90 con 0x7f5a94108ea0 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.092+0000 7f5a937fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 0x7f5a9410b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.092+0000 7f5a937fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 0x7f5a9410b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42834/0 (socket says 172.21.15.132:42834) 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.092+0000 7f5a937fe700 1 -- 172.21.15.132:0/2999725268 learned_addr learned my addr 172.21.15.132:0/2999725268 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.092+0000 7f5a937fe700 1 -- 172.21.15.132:0/2999725268 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9410b7d0 con 0x7f5a94108ea0 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.092+0000 7f5a937fe700 1 --2- 172.21.15.132:0/2999725268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 0x7f5a9410b290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5a880098a0 tx=0x7f5a88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8daf42dd23e2c7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.092+0000 7f5a927fc700 1 -- 172.21.15.132:0/2999725268 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a88004160 con 0x7f5a94108ea0 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.092+0000 7f5a927fc700 1 -- 172.21.15.132:0/2999725268 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5a88004300 con 0x7f5a94108ea0 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.092+0000 7f5a927fc700 1 -- 172.21.15.132:0/2999725268 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a88011af0 con 0x7f5a94108ea0 2024-04-23T18:26:11.741 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.093+0000 7f5a99c68700 1 -- 172.21.15.132:0/2999725268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 msgr2=0x7f5a9410b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.093+0000 7f5a99c68700 1 --2- 172.21.15.132:0/2999725268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 0x7f5a9410b290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5a880098a0 tx=0x7f5a88009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.093+0000 7f5a99c68700 1 -- 172.21.15.132:0/2999725268 shutdown_connections 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.093+0000 7f5a99c68700 1 --2- 172.21.15.132:0/2999725268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 0x7f5a9410b290 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.093+0000 7f5a99c68700 1 -- 172.21.15.132:0/2999725268 >> 172.21.15.132:0/2999725268 conn(0x7f5a94102890 msgr2=0x7f5a94104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.093+0000 7f5a99c68700 1 -- 172.21.15.132:0/2999725268 shutdown_connections 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.093+0000 7f5a99c68700 1 -- 172.21.15.132:0/2999725268 wait complete. 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.094+0000 7f5a99c68700 1 Processor -- start 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.094+0000 7f5a99c68700 1 -- start start 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.094+0000 7f5a99c68700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 0x7f5a9419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.094+0000 7f5a99c68700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9410c790 con 0x7f5a94108ea0 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.094+0000 7f5a937fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 0x7f5a9419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.094+0000 7f5a937fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 0x7f5a9419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42844/0 (socket says 172.21.15.132:42844) 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.094+0000 7f5a937fe700 1 -- 172.21.15.132:0/2865785030 learned_addr learned my addr 172.21.15.132:0/2865785030 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:11.742 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.095+0000 7f5a937fe700 1 -- 172.21.15.132:0/2865785030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9419eb20 con 0x7f5a94108ea0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.095+0000 7f5a937fe700 1 --2- 172.21.15.132:0/2865785030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 0x7f5a9419e5e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009fb0 tx=0x7f5a88011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.095+0000 7f5a90ff9700 1 -- 172.21.15.132:0/2865785030 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a880036a0 con 0x7f5a94108ea0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.095+0000 7f5a90ff9700 1 -- 172.21.15.132:0/2865785030 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5a8801a430 con 0x7f5a94108ea0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.095+0000 7f5a99c68700 1 -- 172.21.15.132:0/2865785030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9419edb0 con 0x7f5a94108ea0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.095+0000 7f5a90ff9700 1 -- 172.21.15.132:0/2865785030 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a88019550 con 0x7f5a94108ea0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.095+0000 7f5a99c68700 1 -- 172.21.15.132:0/2865785030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a941a1aa0 con 0x7f5a94108ea0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.095+0000 7f5a90ff9700 1 -- 172.21.15.132:0/2865785030 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5a88019be0 con 0x7f5a94108ea0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.095+0000 7f5a90ff9700 1 -- 172.21.15.132:0/2865785030 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5a88024730 con 0x7f5a94108ea0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.096+0000 7f5a99c68700 1 -- 172.21.15.132:0/2865785030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a940640a0 con 0x7f5a94108ea0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.099+0000 7f5a90ff9700 1 -- 172.21.15.132:0/2865785030 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5a88043e50 con 0x7f5a94108ea0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.169+0000 7f5a99c68700 1 -- 172.21.15.132:0/2865785030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5a941a1c70 con 0x7f5a94108ea0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.170+0000 7f5a90ff9700 1 -- 172.21.15.132:0/2865785030 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5a88032030 con 0x7f5a94108ea0 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.170+0000 7f5a99c68700 1 -- 172.21.15.132:0/2865785030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 msgr2=0x7f5a9419e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.170+0000 7f5a99c68700 1 --2- 172.21.15.132:0/2865785030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 0x7f5a9419e5e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009fb0 tx=0x7f5a88011940 comp rx=0 tx=0).stop 2024-04-23T18:26:11.743 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.171+0000 7f5a99c68700 1 -- 172.21.15.132:0/2865785030 shutdown_connections 2024-04-23T18:26:11.744 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.171+0000 7f5a99c68700 1 --2- 172.21.15.132:0/2865785030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a94108ea0 0x7f5a9419e5e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:11.744 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.171+0000 7f5a99c68700 1 -- 172.21.15.132:0/2865785030 >> 172.21.15.132:0/2865785030 conn(0x7f5a94102890 msgr2=0x7f5a941033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:11.744 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.171+0000 7f5a99c68700 1 -- 172.21.15.132:0/2865785030 shutdown_connections 2024-04-23T18:26:11.744 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:11.171+0000 7f5a99c68700 1 -- 172.21.15.132:0/2865785030 wait complete. 2024-04-23T18:26:11.744 INFO:teuthology.orchestra.run.smithi132.stdout:mgr not available, waiting (2/15)... 2024-04-23T18:26:12.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:11 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:11.749+0000 7f1bca38a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:26:12.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:11 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:11.949+0000 7f1bca38a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:26:12.793 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:12 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:12.466+0000 7f1bca38a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:26:12.793 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:12 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:12.557+0000 7f1bca38a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:26:13.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:12 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:12.800+0000 7f1bca38a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:26:13.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:12 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:12.932+0000 7f1bca38a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:26:13.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:13 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:13.025+0000 7f1bca38a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:26:13.466 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:13 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:13.142+0000 7f1bca38a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:13.466 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:13 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:13.473+0000 7f1bca38a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:26:14.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:14 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:14.080+0000 7f1bca38a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:26:14.612 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:14 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:14.307+0000 7f1bca38a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:26:14.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:14 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3560392755' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:26:14.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:14 smithi132 ceph-mon[29178]: Activating manager daemon x 2024-04-23T18:26:14.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:14 smithi132 ceph-mon[29178]: mgrmap e2: x(active, starting, since 0.00320151s) 2024-04-23T18:26:14.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:14 smithi132 ceph-mon[29178]: from='mgr.14100 172.21.15.132:0/3241582700' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:26:14.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:14 smithi132 ceph-mon[29178]: from='mgr.14100 172.21.15.132:0/3241582700' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:26:14.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:14 smithi132 ceph-mon[29178]: from='mgr.14100 172.21.15.132:0/3241582700' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:26:14.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:14 smithi132 ceph-mon[29178]: from='mgr.14100 172.21.15.132:0/3241582700' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:26:14.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:14 smithi132 ceph-mon[29178]: from='mgr.14100 172.21.15.132:0/3241582700' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:26:14.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:14 smithi132 ceph-mon[29178]: Manager daemon x is now available 2024-04-23T18:26:14.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:26:14.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:26:14.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-23T18:26:14.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:26:14.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:14.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:26:14.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:26:14.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:14.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:26:14.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:14.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:26:14.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:26:14.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:26:14.908 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:26:14.908 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:26:14.908 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:26:14.908 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:14.908 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:26:14.908 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:14.909 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:26:14.910 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T18:26:14.910 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:26:14.910 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:26:14.910 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:26:14.910 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:26:14.910 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:26:14.910 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:26:14.911 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:26:14.911 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:14.911 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:26:14.911 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:14.911 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:26:00.784990+0000", 2024-04-23T18:26:14.911 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:26:14.911 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:14.911 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:26:14.911 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:26:14.911 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.196+0000 7fd2880d4700 1 Processor -- start 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.196+0000 7fd2880d4700 1 -- start start 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.196+0000 7fd2880d4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 0x7fd280108d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.196+0000 7fd2880d4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801092e0 con 0x7fd2801088f0 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.196+0000 7fd285e70700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 0x7fd280108d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.196+0000 7fd285e70700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 0x7fd280108d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42854/0 (socket says 172.21.15.132:42854) 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.196+0000 7fd285e70700 1 -- 172.21.15.132:0/1539320313 learned_addr learned my addr 172.21.15.132:0/1539320313 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.196+0000 7fd285e70700 1 -- 172.21.15.132:0/1539320313 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2801075a0 con 0x7fd2801088f0 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.197+0000 7fd285e70700 1 --2- 172.21.15.132:0/1539320313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 0x7fd280108d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0098a0 tx=0x7fd27c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6556b417a939c57c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.197+0000 7fd284e6e700 1 -- 172.21.15.132:0/1539320313 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd27c004160 con 0x7fd2801088f0 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.197+0000 7fd284e6e700 1 -- 172.21.15.132:0/1539320313 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd27c004300 con 0x7fd2801088f0 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.197+0000 7fd284e6e700 1 -- 172.21.15.132:0/1539320313 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd27c011af0 con 0x7fd2801088f0 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.197+0000 7fd2880d4700 1 -- 172.21.15.132:0/1539320313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 msgr2=0x7fd280108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.197+0000 7fd2880d4700 1 --2- 172.21.15.132:0/1539320313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 0x7fd280108d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0098a0 tx=0x7fd27c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.197+0000 7fd2880d4700 1 -- 172.21.15.132:0/1539320313 shutdown_connections 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.198+0000 7fd2880d4700 1 --2- 172.21.15.132:0/1539320313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 0x7fd280108d10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.912 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.198+0000 7fd2880d4700 1 -- 172.21.15.132:0/1539320313 >> 172.21.15.132:0/1539320313 conn(0x7fd280102030 msgr2=0x7fd280104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.198+0000 7fd2880d4700 1 -- 172.21.15.132:0/1539320313 shutdown_connections 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.198+0000 7fd2880d4700 1 -- 172.21.15.132:0/1539320313 wait complete. 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.198+0000 7fd2880d4700 1 Processor -- start 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.198+0000 7fd2880d4700 1 -- start start 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.198+0000 7fd2880d4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 0x7fd280106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.198+0000 7fd2880d4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd28010b960 con 0x7fd2801088f0 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.198+0000 7fd285e70700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 0x7fd280106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.198+0000 7fd285e70700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 0x7fd280106f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42856/0 (socket says 172.21.15.132:42856) 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.198+0000 7fd285e70700 1 -- 172.21.15.132:0/3560392755 learned_addr learned my addr 172.21.15.132:0/3560392755 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.199+0000 7fd285e70700 1 -- 172.21.15.132:0/3560392755 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28019f120 con 0x7fd2801088f0 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.199+0000 7fd285e70700 1 --2- 172.21.15.132:0/3560392755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 0x7fd280106f10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0117b0 tx=0x7fd27c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.199+0000 7fd276ffd700 1 -- 172.21.15.132:0/3560392755 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd27c003af0 con 0x7fd2801088f0 2024-04-23T18:26:14.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.199+0000 7fd276ffd700 1 -- 172.21.15.132:0/3560392755 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd27c020070 con 0x7fd2801088f0 2024-04-23T18:26:14.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.199+0000 7fd2880d4700 1 -- 172.21.15.132:0/3560392755 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28019f2f0 con 0x7fd2801088f0 2024-04-23T18:26:14.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.199+0000 7fd276ffd700 1 -- 172.21.15.132:0/3560392755 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd27c019630 con 0x7fd2801088f0 2024-04-23T18:26:14.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.199+0000 7fd2880d4700 1 -- 172.21.15.132:0/3560392755 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28019f690 con 0x7fd2801088f0 2024-04-23T18:26:14.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.199+0000 7fd276ffd700 1 -- 172.21.15.132:0/3560392755 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd27c019c80 con 0x7fd2801088f0 2024-04-23T18:26:14.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.199+0000 7fd276ffd700 1 -- 172.21.15.132:0/3560392755 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd27c00b720 con 0x7fd2801088f0 2024-04-23T18:26:14.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.200+0000 7fd2880d4700 1 -- 172.21.15.132:0/3560392755 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2800640a0 con 0x7fd2801088f0 2024-04-23T18:26:14.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.204+0000 7fd276ffd700 1 -- 172.21.15.132:0/3560392755 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd27c0042a0 con 0x7fd2801088f0 2024-04-23T18:26:14.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.276+0000 7fd2880d4700 1 -- 172.21.15.132:0/3560392755 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd28019f990 con 0x7fd2801088f0 2024-04-23T18:26:14.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.276+0000 7fd276ffd700 1 -- 172.21.15.132:0/3560392755 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd27c01a9c0 con 0x7fd2801088f0 2024-04-23T18:26:14.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.277+0000 7fd2880d4700 1 -- 172.21.15.132:0/3560392755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 msgr2=0x7fd280106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:14.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.277+0000 7fd2880d4700 1 --2- 172.21.15.132:0/3560392755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 0x7fd280106f10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0117b0 tx=0x7fd27c011b70 comp rx=0 tx=0).stop 2024-04-23T18:26:14.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.277+0000 7fd2880d4700 1 -- 172.21.15.132:0/3560392755 shutdown_connections 2024-04-23T18:26:14.915 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.277+0000 7fd2880d4700 1 --2- 172.21.15.132:0/3560392755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2801088f0 0x7fd280106f10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:14.915 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.277+0000 7fd2880d4700 1 -- 172.21.15.132:0/3560392755 >> 172.21.15.132:0/3560392755 conn(0x7fd280102030 msgr2=0x7fd280102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:14.915 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.278+0000 7fd2880d4700 1 -- 172.21.15.132:0/3560392755 shutdown_connections 2024-04-23T18:26:14.915 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:14.278+0000 7fd2880d4700 1 -- 172.21.15.132:0/3560392755 wait complete. 2024-04-23T18:26:14.915 INFO:teuthology.orchestra.run.smithi132.stdout:mgr not available, waiting (3/15)... 2024-04-23T18:26:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:15 smithi132 ceph-mon[29178]: from='mgr.14100 172.21.15.132:0/3241582700' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:26:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:15 smithi132 ceph-mon[29178]: from='mgr.14100 172.21.15.132:0/3241582700' entity='mgr.x' 2024-04-23T18:26:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:15 smithi132 ceph-mon[29178]: from='mgr.14100 172.21.15.132:0/3241582700' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:26:15.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:15 smithi132 ceph-mon[29178]: from='mgr.14100 172.21.15.132:0/3241582700' entity='mgr.x' 2024-04-23T18:26:15.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:15 smithi132 ceph-mon[29178]: from='mgr.14100 172.21.15.132:0/3241582700' entity='mgr.x' 2024-04-23T18:26:15.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:15 smithi132 ceph-mon[29178]: mgrmap e3: x(active, since 1.00649s) 2024-04-23T18:26:17.592 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:17 smithi132 ceph-mon[29178]: mgrmap e4: x(active, since 2s) 2024-04-23T18:26:18.213 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:18.213 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:26:18.213 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:26:18.213 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:26:18.213 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:26:18.213 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:26:18.213 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:26:18.213 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:18.213 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:26:18.213 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:26:18.213 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:26:18.213 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:26:18.214 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:26:18.222 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:26:18.223 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:26:18.224 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:26:00.784990+0000", 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.309+0000 7f8c2da0f700 1 Processor -- start 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.309+0000 7f8c2da0f700 1 -- start start 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.309+0000 7f8c2da0f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 0x7f8c2807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.309+0000 7f8c2da0f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2807c8f0 con 0x7f8c2807de70 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.309+0000 7f8c26ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 0x7f8c2807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.310+0000 7f8c26ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 0x7f8c2807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42902/0 (socket says 172.21.15.132:42902) 2024-04-23T18:26:18.225 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.310+0000 7f8c26ffd700 1 -- 172.21.15.132:0/2368148015 learned_addr learned my addr 172.21.15.132:0/2368148015 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.310+0000 7f8c26ffd700 1 -- 172.21.15.132:0/2368148015 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2807ca70 con 0x7f8c2807de70 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.310+0000 7f8c26ffd700 1 --2- 172.21.15.132:0/2368148015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 0x7f8c2807c320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c180098a0 tx=0x7f8c18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3b1bfbdc6b6ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.310+0000 7f8c25ffb700 1 -- 172.21.15.132:0/2368148015 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c18004160 con 0x7f8c2807de70 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.310+0000 7f8c25ffb700 1 -- 172.21.15.132:0/2368148015 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8c18004300 con 0x7f8c2807de70 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.310+0000 7f8c25ffb700 1 -- 172.21.15.132:0/2368148015 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c18011af0 con 0x7f8c2807de70 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.311+0000 7f8c2da0f700 1 -- 172.21.15.132:0/2368148015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 msgr2=0x7f8c2807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.311+0000 7f8c2da0f700 1 --2- 172.21.15.132:0/2368148015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 0x7f8c2807c320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c180098a0 tx=0x7f8c18009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.311+0000 7f8c2da0f700 1 -- 172.21.15.132:0/2368148015 shutdown_connections 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.311+0000 7f8c2da0f700 1 --2- 172.21.15.132:0/2368148015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 0x7f8c2807c320 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.311+0000 7f8c2da0f700 1 -- 172.21.15.132:0/2368148015 >> 172.21.15.132:0/2368148015 conn(0x7f8c281039f0 msgr2=0x7f8c28105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.311+0000 7f8c2da0f700 1 -- 172.21.15.132:0/2368148015 shutdown_connections 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.311+0000 7f8c2da0f700 1 -- 172.21.15.132:0/2368148015 wait complete. 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.311+0000 7f8c2da0f700 1 Processor -- start 2024-04-23T18:26:18.226 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c2da0f700 1 -- start start 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c2da0f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 0x7f8c2819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c2da0f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2810e940 con 0x7f8c2807de70 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c26ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 0x7f8c2819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c26ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 0x7f8c2819e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42910/0 (socket says 172.21.15.132:42910) 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c26ffd700 1 -- 172.21.15.132:0/1171502909 learned_addr learned my addr 172.21.15.132:0/1171502909 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c26ffd700 1 -- 172.21.15.132:0/1171502909 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2819ea60 con 0x7f8c2807de70 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c26ffd700 1 --2- 172.21.15.132:0/1171502909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 0x7f8c2819e520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8c18011ac0 tx=0x7f8c18011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c2ca0d700 1 -- 172.21.15.132:0/1171502909 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c18003b20 con 0x7f8c2807de70 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c2ca0d700 1 -- 172.21.15.132:0/1171502909 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8c1801a8c0 con 0x7f8c2807de70 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c2ca0d700 1 -- 172.21.15.132:0/1171502909 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c180248d0 con 0x7f8c2807de70 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c2da0f700 1 -- 172.21.15.132:0/1171502909 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2819ecf0 con 0x7f8c2807de70 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.312+0000 7f8c2da0f700 1 -- 172.21.15.132:0/1171502909 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c281a19e0 con 0x7f8c2807de70 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.313+0000 7f8c2ca0d700 1 -- 172.21.15.132:0/1171502909 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f8c18022020 con 0x7f8c2807de70 2024-04-23T18:26:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.313+0000 7f8c2da0f700 1 -- 172.21.15.132:0/1171502909 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c280640a0 con 0x7f8c2807de70 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.313+0000 7f8c2ca0d700 1 --2- 172.21.15.132:0/1171502909 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f8c100395b0 0x7f8c1003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.314+0000 7f8c2ca0d700 1 -- 172.21.15.132:0/1171502909 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8c1804d490 con 0x7f8c2807de70 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.314+0000 7f8c267fc700 1 --2- 172.21.15.132:0/1171502909 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f8c100395b0 0x7f8c1003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.314+0000 7f8c267fc700 1 --2- 172.21.15.132:0/1171502909 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f8c100395b0 0x7f8c1003ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c006fd0 tx=0x7f8c1c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.319+0000 7f8c2ca0d700 1 -- 172.21.15.132:0/1171502909 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c1801ab20 con 0x7f8c2807de70 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.599+0000 7f8c2da0f700 1 -- 172.21.15.132:0/1171502909 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8c281a1bb0 con 0x7f8c2807de70 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.600+0000 7f8c2ca0d700 1 -- 172.21.15.132:0/1171502909 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8c281a1bb0 con 0x7f8c2807de70 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.602+0000 7f8c2da0f700 1 -- 172.21.15.132:0/1171502909 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f8c100395b0 msgr2=0x7f8c1003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.602+0000 7f8c2da0f700 1 --2- 172.21.15.132:0/1171502909 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f8c100395b0 0x7f8c1003ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c006fd0 tx=0x7f8c1c006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.602+0000 7f8c2da0f700 1 -- 172.21.15.132:0/1171502909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 msgr2=0x7f8c2819e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.602+0000 7f8c2da0f700 1 --2- 172.21.15.132:0/1171502909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 0x7f8c2819e520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8c18011ac0 tx=0x7f8c18011ba0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.602+0000 7f8c2da0f700 1 -- 172.21.15.132:0/1171502909 shutdown_connections 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.602+0000 7f8c2da0f700 1 --2- 172.21.15.132:0/1171502909 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f8c100395b0 0x7f8c1003ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.602+0000 7f8c2da0f700 1 --2- 172.21.15.132:0/1171502909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c2807de70 0x7f8c2819e520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.602+0000 7f8c2da0f700 1 -- 172.21.15.132:0/1171502909 >> 172.21.15.132:0/1171502909 conn(0x7f8c281039f0 msgr2=0x7f8c281045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.602+0000 7f8c2da0f700 1 -- 172.21.15.132:0/1171502909 shutdown_connections 2024-04-23T18:26:18.229 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:17.602+0000 7f8c2da0f700 1 -- 172.21.15.132:0/1171502909 wait complete. 2024-04-23T18:26:18.229 INFO:teuthology.orchestra.run.smithi132.stdout:mgr is available 2024-04-23T18:26:18.511 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:18 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1171502909' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:26:19.349 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout fsid = c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T18:26:19.350 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.616+0000 7fac57df8700 1 Processor -- start 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.616+0000 7fac57df8700 1 -- start start 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.616+0000 7fac57df8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 0x7fac50108d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.616+0000 7fac57df8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fac50109340 con 0x7fac50108950 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.616+0000 7fac55b94700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 0x7fac50108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.616+0000 7fac55b94700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 0x7fac50108d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42918/0 (socket says 172.21.15.132:42918) 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.616+0000 7fac55b94700 1 -- 172.21.15.132:0/398273028 learned_addr learned my addr 172.21.15.132:0/398273028 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.617+0000 7fac55b94700 1 -- 172.21.15.132:0/398273028 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac50107560 con 0x7fac50108950 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.617+0000 7fac55b94700 1 --2- 172.21.15.132:0/398273028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 0x7fac50108d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0098a0 tx=0x7fac4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48166445cbc9dd21 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:19.351 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.617+0000 7fac54b92700 1 -- 172.21.15.132:0/398273028 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac4c004160 con 0x7fac50108950 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.617+0000 7fac54b92700 1 -- 172.21.15.132:0/398273028 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac4c004300 con 0x7fac50108950 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.617+0000 7fac54b92700 1 -- 172.21.15.132:0/398273028 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac4c011af0 con 0x7fac50108950 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.617+0000 7fac57df8700 1 -- 172.21.15.132:0/398273028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 msgr2=0x7fac50108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.617+0000 7fac57df8700 1 --2- 172.21.15.132:0/398273028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 0x7fac50108d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0098a0 tx=0x7fac4c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.617+0000 7fac57df8700 1 -- 172.21.15.132:0/398273028 shutdown_connections 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.617+0000 7fac57df8700 1 --2- 172.21.15.132:0/398273028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 0x7fac50108d70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.617+0000 7fac57df8700 1 -- 172.21.15.132:0/398273028 >> 172.21.15.132:0/398273028 conn(0x7fac50102150 msgr2=0x7fac50104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.618+0000 7fac57df8700 1 -- 172.21.15.132:0/398273028 shutdown_connections 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.618+0000 7fac57df8700 1 -- 172.21.15.132:0/398273028 wait complete. 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.618+0000 7fac57df8700 1 Processor -- start 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.618+0000 7fac57df8700 1 -- start start 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.618+0000 7fac57df8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 0x7fac50192490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.618+0000 7fac57df8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fac5010b920 con 0x7fac50108950 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.618+0000 7fac55b94700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 0x7fac50192490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.618+0000 7fac55b94700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 0x7fac50192490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42924/0 (socket says 172.21.15.132:42924) 2024-04-23T18:26:19.352 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.619+0000 7fac55b94700 1 -- 172.21.15.132:0/4160460589 learned_addr learned my addr 172.21.15.132:0/4160460589 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.619+0000 7fac55b94700 1 -- 172.21.15.132:0/4160460589 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac501929d0 con 0x7fac50108950 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.619+0000 7fac55b94700 1 --2- 172.21.15.132:0/4160460589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 0x7fac50192490 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0117b0 tx=0x7fac4c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.619+0000 7fac46ffd700 1 -- 172.21.15.132:0/4160460589 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac4c003af0 con 0x7fac50108950 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.619+0000 7fac46ffd700 1 -- 172.21.15.132:0/4160460589 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac4c020070 con 0x7fac50108950 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.619+0000 7fac57df8700 1 -- 172.21.15.132:0/4160460589 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac50190c10 con 0x7fac50108950 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.619+0000 7fac46ffd700 1 -- 172.21.15.132:0/4160460589 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac4c019630 con 0x7fac50108950 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.619+0000 7fac57df8700 1 -- 172.21.15.132:0/4160460589 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac501910f0 con 0x7fac50108950 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.620+0000 7fac46ffd700 1 -- 172.21.15.132:0/4160460589 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fac4c0198f0 con 0x7fac50108950 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.620+0000 7fac46ffd700 1 --2- 172.21.15.132:0/4160460589 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7fac3c039560 0x7fac3c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.620+0000 7fac57df8700 1 -- 172.21.15.132:0/4160460589 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac500640a0 con 0x7fac50108950 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.620+0000 7fac46ffd700 1 -- 172.21.15.132:0/4160460589 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac4c04dd10 con 0x7fac50108950 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.620+0000 7fac55393700 1 --2- 172.21.15.132:0/4160460589 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7fac3c039560 0x7fac3c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.620+0000 7fac55393700 1 --2- 172.21.15.132:0/4160460589 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7fac3c039560 0x7fac3c03ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac40006fd0 tx=0x7fac40006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:19.353 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.625+0000 7fac46ffd700 1 -- 172.21.15.132:0/4160460589 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac4c0042a0 con 0x7fac50108950 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.821+0000 7fac57df8700 1 -- 172.21.15.132:0/4160460589 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fac501961f0 con 0x7fac50108950 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.822+0000 7fac46ffd700 1 -- 172.21.15.132:0/4160460589 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fac4c04d730 con 0x7fac50108950 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.824+0000 7fac57df8700 1 -- 172.21.15.132:0/4160460589 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7fac3c039560 msgr2=0x7fac3c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.824+0000 7fac57df8700 1 --2- 172.21.15.132:0/4160460589 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7fac3c039560 0x7fac3c03ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac40006fd0 tx=0x7fac40006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.824+0000 7fac57df8700 1 -- 172.21.15.132:0/4160460589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 msgr2=0x7fac50192490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.824+0000 7fac57df8700 1 --2- 172.21.15.132:0/4160460589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 0x7fac50192490 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0117b0 tx=0x7fac4c011b70 comp rx=0 tx=0).stop 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.824+0000 7fac57df8700 1 -- 172.21.15.132:0/4160460589 shutdown_connections 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.824+0000 7fac57df8700 1 --2- 172.21.15.132:0/4160460589 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7fac3c039560 0x7fac3c03ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.824+0000 7fac57df8700 1 --2- 172.21.15.132:0/4160460589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac50108950 0x7fac50192490 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.824+0000 7fac57df8700 1 -- 172.21.15.132:0/4160460589 >> 172.21.15.132:0/4160460589 conn(0x7fac50102150 msgr2=0x7fac50103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.824+0000 7fac57df8700 1 -- 172.21.15.132:0/4160460589 shutdown_connections 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:18.824+0000 7fac57df8700 1 -- 172.21.15.132:0/4160460589 wait complete. 2024-04-23T18:26:19.354 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling cephadm module... 2024-04-23T18:26:19.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:19 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/4160460589' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T18:26:20.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:20 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3954081217' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-23T18:26:20.612 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:20 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: ignoring --setuser ceph since I am not root 2024-04-23T18:26:20.612 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:20 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: ignoring --setgroup ceph since I am not root 2024-04-23T18:26:20.612 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:20 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:20.509+0000 7f769051f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:26:20.912 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:20 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:20.696+0000 7f769051f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:20.912 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:20 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:20.798+0000 7f769051f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:26:20.920 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.740+0000 7f28a00a4700 1 Processor -- start 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.740+0000 7f28a00a4700 1 -- start start 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.740+0000 7f28a00a4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 0x7f28981088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.740+0000 7f28a00a4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2898108e90 con 0x7f28981084a0 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.740+0000 7f289de40700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 0x7f28981088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.740+0000 7f289de40700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 0x7f28981088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57450/0 (socket says 172.21.15.132:57450) 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.740+0000 7f289de40700 1 -- 172.21.15.132:0/3332183063 learned_addr learned my addr 172.21.15.132:0/3332183063 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.741+0000 7f289de40700 1 -- 172.21.15.132:0/3332183063 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2898109650 con 0x7f28981084a0 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.741+0000 7f289de40700 1 --2- 172.21.15.132:0/3332183063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 0x7f28981088c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f28940098a0 tx=0x7f2894009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1192dd033bc04af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.741+0000 7f289ce3e700 1 -- 172.21.15.132:0/3332183063 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2894004160 con 0x7f28981084a0 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.741+0000 7f289ce3e700 1 -- 172.21.15.132:0/3332183063 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2894004300 con 0x7f28981084a0 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.741+0000 7f289ce3e700 1 -- 172.21.15.132:0/3332183063 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2894011af0 con 0x7f28981084a0 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.741+0000 7f28a00a4700 1 -- 172.21.15.132:0/3332183063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 msgr2=0x7f28981088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.921 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.741+0000 7f28a00a4700 1 --2- 172.21.15.132:0/3332183063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 0x7f28981088c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f28940098a0 tx=0x7f2894009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.742+0000 7f28a00a4700 1 -- 172.21.15.132:0/3332183063 shutdown_connections 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.742+0000 7f28a00a4700 1 --2- 172.21.15.132:0/3332183063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 0x7f28981088c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.742+0000 7f28a00a4700 1 -- 172.21.15.132:0/3332183063 >> 172.21.15.132:0/3332183063 conn(0x7f2898103ac0 msgr2=0x7f2898105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.742+0000 7f28a00a4700 1 -- 172.21.15.132:0/3332183063 shutdown_connections 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.742+0000 7f28a00a4700 1 -- 172.21.15.132:0/3332183063 wait complete. 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.742+0000 7f28a00a4700 1 Processor -- start 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.742+0000 7f28a00a4700 1 -- start start 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.742+0000 7f28a00a4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 0x7f289819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.742+0000 7f28a00a4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f289810ca00 con 0x7f28981084a0 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.742+0000 7f289de40700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 0x7f289819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.742+0000 7f289de40700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 0x7f289819c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57454/0 (socket says 172.21.15.132:57454) 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.742+0000 7f289de40700 1 -- 172.21.15.132:0/3954081217 learned_addr learned my addr 172.21.15.132:0/3954081217 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.743+0000 7f289de40700 1 -- 172.21.15.132:0/3954081217 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289819c920 con 0x7f28981084a0 2024-04-23T18:26:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.743+0000 7f289de40700 1 --2- 172.21.15.132:0/3954081217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 0x7f289819c3e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f28940117b0 tx=0x7f2894011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.743+0000 7f288effd700 1 -- 172.21.15.132:0/3954081217 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2894003af0 con 0x7f28981084a0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.743+0000 7f28a00a4700 1 -- 172.21.15.132:0/3954081217 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289819cbb0 con 0x7f28981084a0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.743+0000 7f288effd700 1 -- 172.21.15.132:0/3954081217 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2894020070 con 0x7f28981084a0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.743+0000 7f288effd700 1 -- 172.21.15.132:0/3954081217 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2894019630 con 0x7f28981084a0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.743+0000 7f28a00a4700 1 -- 172.21.15.132:0/3954081217 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289819f8a0 con 0x7f28981084a0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.744+0000 7f288effd700 1 -- 172.21.15.132:0/3954081217 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f28940198f0 con 0x7f28981084a0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.744+0000 7f28a00a4700 1 -- 172.21.15.132:0/3954081217 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28980640a0 con 0x7f28981084a0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.744+0000 7f288effd700 1 --2- 172.21.15.132:0/3954081217 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f2884039560 0x7f288403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.744+0000 7f288effd700 1 -- 172.21.15.132:0/3954081217 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f289404dc60 con 0x7f28981084a0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.744+0000 7f289d63f700 1 --2- 172.21.15.132:0/3954081217 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f2884039560 0x7f288403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.744+0000 7f289d63f700 1 --2- 172.21.15.132:0/3954081217 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f2884039560 0x7f288403ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2888006fd0 tx=0x7f2888006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:19.749+0000 7f288effd700 1 -- 172.21.15.132:0/3954081217 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2894023370 con 0x7f28981084a0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.003+0000 7f28a00a4700 1 -- 172.21.15.132:0/3954081217 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f289819fa70 con 0x7f28981084a0 2024-04-23T18:26:20.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.334+0000 7f288effd700 1 -- 172.21.15.132:0/3954081217 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f289819fa70 con 0x7f28981084a0 2024-04-23T18:26:20.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.336+0000 7f288effd700 1 -- 172.21.15.132:0/3954081217 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f289400b430 con 0x7f28981084a0 2024-04-23T18:26:20.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.337+0000 7f28a00a4700 1 -- 172.21.15.132:0/3954081217 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f2884039560 msgr2=0x7f288403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.337+0000 7f28a00a4700 1 --2- 172.21.15.132:0/3954081217 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f2884039560 0x7f288403ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2888006fd0 tx=0x7f2888006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:20.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.337+0000 7f28a00a4700 1 -- 172.21.15.132:0/3954081217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 msgr2=0x7f289819c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.337+0000 7f28a00a4700 1 --2- 172.21.15.132:0/3954081217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 0x7f289819c3e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f28940117b0 tx=0x7f2894011b70 comp rx=0 tx=0).stop 2024-04-23T18:26:20.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.337+0000 7f28a00a4700 1 -- 172.21.15.132:0/3954081217 shutdown_connections 2024-04-23T18:26:20.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.337+0000 7f28a00a4700 1 --2- 172.21.15.132:0/3954081217 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f2884039560 0x7f288403ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.337+0000 7f28a00a4700 1 --2- 172.21.15.132:0/3954081217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28981084a0 0x7f289819c3e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.337+0000 7f28a00a4700 1 -- 172.21.15.132:0/3954081217 >> 172.21.15.132:0/3954081217 conn(0x7f2898103ac0 msgr2=0x7f2898104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.337+0000 7f28a00a4700 1 -- 172.21.15.132:0/3954081217 shutdown_connections 2024-04-23T18:26:20.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:20.337+0000 7f28a00a4700 1 -- 172.21.15.132:0/3954081217 wait complete. 2024-04-23T18:26:21.196 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:21 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:21.113+0000 7f769051f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:26:21.540 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:21 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:21.204+0000 7f769051f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:26:21.540 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:21 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3954081217' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-23T18:26:21.540 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:21 smithi132 ceph-mon[29178]: mgrmap e5: x(active, since 6s) 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.339+0000 7f30bb168700 1 Processor -- start 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.339+0000 7f30bb168700 1 -- start start 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.339+0000 7f30bb168700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 0x7f30b407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.339+0000 7f30bb168700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f30b407c8f0 con 0x7f30b407de70 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.339+0000 7f30b8f04700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 0x7f30b407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.339+0000 7f30b8f04700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 0x7f30b407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57494/0 (socket says 172.21.15.132:57494) 2024-04-23T18:26:22.236 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.339+0000 7f30b8f04700 1 -- 172.21.15.132:0/959172023 learned_addr learned my addr 172.21.15.132:0/959172023 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.339+0000 7f30b8f04700 1 -- 172.21.15.132:0/959172023 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b407ca70 con 0x7f30b407de70 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.340+0000 7f30b8f04700 1 --2- 172.21.15.132:0/959172023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 0x7f30b407c320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f30a80098a0 tx=0x7f30a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa35bdae339b9a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.342+0000 7f30b37fe700 1 -- 172.21.15.132:0/959172023 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30a8004160 con 0x7f30b407de70 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.342+0000 7f30b37fe700 1 -- 172.21.15.132:0/959172023 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f30a8004300 con 0x7f30b407de70 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.342+0000 7f30b37fe700 1 -- 172.21.15.132:0/959172023 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30a8011af0 con 0x7f30b407de70 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.342+0000 7f30bb168700 1 -- 172.21.15.132:0/959172023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 msgr2=0x7f30b407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.342+0000 7f30bb168700 1 --2- 172.21.15.132:0/959172023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 0x7f30b407c320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f30a80098a0 tx=0x7f30a8009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.342+0000 7f30bb168700 1 -- 172.21.15.132:0/959172023 shutdown_connections 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.342+0000 7f30bb168700 1 --2- 172.21.15.132:0/959172023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 0x7f30b407c320 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.342+0000 7f30bb168700 1 -- 172.21.15.132:0/959172023 >> 172.21.15.132:0/959172023 conn(0x7f30b41039f0 msgr2=0x7f30b4105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.342+0000 7f30bb168700 1 -- 172.21.15.132:0/959172023 shutdown_connections 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.342+0000 7f30bb168700 1 -- 172.21.15.132:0/959172023 wait complete. 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.343+0000 7f30bb168700 1 Processor -- start 2024-04-23T18:26:22.237 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.343+0000 7f30bb168700 1 -- start start 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.343+0000 7f30bb168700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 0x7f30b419e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.343+0000 7f30bb168700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f30b410e960 con 0x7f30b407de70 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.343+0000 7f30b8f04700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 0x7f30b419e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.343+0000 7f30b8f04700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 0x7f30b419e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57508/0 (socket says 172.21.15.132:57508) 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.343+0000 7f30b8f04700 1 -- 172.21.15.132:0/1554287473 learned_addr learned my addr 172.21.15.132:0/1554287473 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.344+0000 7f30b8f04700 1 -- 172.21.15.132:0/1554287473 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b419ea60 con 0x7f30b407de70 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.344+0000 7f30b8f04700 1 --2- 172.21.15.132:0/1554287473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 0x7f30b419e520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f30a8011ac0 tx=0x7f30a8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.344+0000 7f30b1ffb700 1 -- 172.21.15.132:0/1554287473 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30a8003b20 con 0x7f30b407de70 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.344+0000 7f30b1ffb700 1 -- 172.21.15.132:0/1554287473 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f30a801a8c0 con 0x7f30b407de70 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.344+0000 7f30bb168700 1 -- 172.21.15.132:0/1554287473 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b419ecf0 con 0x7f30b407de70 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.344+0000 7f30b1ffb700 1 -- 172.21.15.132:0/1554287473 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30a80248d0 con 0x7f30b407de70 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.344+0000 7f30bb168700 1 -- 172.21.15.132:0/1554287473 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b41a19e0 con 0x7f30b407de70 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.345+0000 7f30bb168700 1 -- 172.21.15.132:0/1554287473 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b40640a0 con 0x7f30b407de70 2024-04-23T18:26:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.345+0000 7f30b1ffb700 1 -- 172.21.15.132:0/1554287473 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f30a8022020 con 0x7f30b407de70 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.345+0000 7f30b1ffb700 1 --2- 172.21.15.132:0/1554287473 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f30a40395b0 0x7f30a403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.345+0000 7f30b1ffb700 1 -- 172.21.15.132:0/1554287473 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f30a804d580 con 0x7f30b407de70 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.345+0000 7f30b3fff700 1 -- 172.21.15.132:0/1554287473 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f30a40395b0 msgr2=0x7f30a403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/2811136616 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.346+0000 7f30b3fff700 1 --2- 172.21.15.132:0/1554287473 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f30a40395b0 0x7f30a403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.350+0000 7f30b1ffb700 1 -- 172.21.15.132:0/1554287473 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30a802cdf0 con 0x7f30b407de70 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.547+0000 7f30b3fff700 1 -- 172.21.15.132:0/1554287473 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f30a40395b0 msgr2=0x7f30a403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/2811136616 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.547+0000 7f30b3fff700 1 --2- 172.21.15.132:0/1554287473 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f30a40395b0 0x7f30a403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.648+0000 7f30bb168700 1 -- 172.21.15.132:0/1554287473 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f30b41a1bb0 con 0x7f30b407de70 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.649+0000 7f30b1ffb700 1 -- 172.21.15.132:0/1554287473 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f30b41a1bb0 con 0x7f30b407de70 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.652+0000 7f30bb168700 1 -- 172.21.15.132:0/1554287473 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f30a40395b0 msgr2=0x7f30a403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.652+0000 7f30bb168700 1 --2- 172.21.15.132:0/1554287473 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f30a40395b0 0x7f30a403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.652+0000 7f30bb168700 1 -- 172.21.15.132:0/1554287473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 msgr2=0x7f30b419e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.652+0000 7f30bb168700 1 --2- 172.21.15.132:0/1554287473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 0x7f30b419e520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f30a8011ac0 tx=0x7f30a8011ba0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.652+0000 7f30bb168700 1 -- 172.21.15.132:0/1554287473 shutdown_connections 2024-04-23T18:26:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.652+0000 7f30bb168700 1 --2- 172.21.15.132:0/1554287473 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f30a40395b0 0x7f30a403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.652+0000 7f30bb168700 1 --2- 172.21.15.132:0/1554287473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b407de70 0x7f30b419e520 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.652+0000 7f30bb168700 1 -- 172.21.15.132:0/1554287473 >> 172.21.15.132:0/1554287473 conn(0x7f30b41039f0 msgr2=0x7f30b4104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.653+0000 7f30bb168700 1 -- 172.21.15.132:0/1554287473 shutdown_connections 2024-04-23T18:26:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:21.653+0000 7f30bb168700 1 -- 172.21.15.132:0/1554287473 wait complete. 2024-04-23T18:26:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for the mgr to restart... 2024-04-23T18:26:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for mgr epoch 5... 2024-04-23T18:26:22.583 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:22 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1554287473' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T18:26:22.583 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:22 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:22.415+0000 7f769051f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:26:22.847 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:22 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:22.590+0000 7f769051f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:22.847 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:22 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:22.763+0000 7f769051f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:26:23.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:23 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:23.048+0000 7f769051f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:26:23.481 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:23 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:23.140+0000 7f769051f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:26:23.481 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:23 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:23.344+0000 7f769051f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:26:23.735 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:23 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:23.538+0000 7f769051f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:24.362 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:24 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:24.165+0000 7f769051f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:26:24.862 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:24 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:24.595+0000 7f769051f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:26:24.862 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:24 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:24.694+0000 7f769051f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:26:25.362 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:24 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:24.895+0000 7f769051f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:26:25.741 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:25 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:25.415+0000 7f769051f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:26:25.741 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:25 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:25.506+0000 7f769051f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:26:26.082 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:25 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:25.747+0000 7f769051f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:26:26.082 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:25 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:25.878+0000 7f769051f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:26:26.082 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:25 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:25.972+0000 7f769051f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:26:26.082 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:26 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:26.087+0000 7f769051f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:26.862 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:26 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:26.390+0000 7f769051f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:26:27.240 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:26 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:26.981+0000 7f769051f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:26:27.240 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:27.193+0000 7f769051f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:26:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: Active manager daemon x restarted 2024-04-23T18:26:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: Activating manager daemon x 2024-04-23T18:26:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: osdmap e2: 0 total, 0 up, 0 in 2024-04-23T18:26:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: mgrmap e6: x(active, starting, since 0.00317572s) 2024-04-23T18:26:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:26:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:26:27.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:26:27.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:26:27.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:26:27.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: Manager daemon x is now available 2024-04-23T18:26:27.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:27.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:27.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:27.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:27.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:27 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:26:28.458 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:28 smithi132 ceph-mon[29178]: Found migration_current of "None". Setting to last migration. 2024-04-23T18:26:28.458 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:28 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:26:28.458 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:28 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:28.458 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:28 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:28.458 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:28 smithi132 ceph-mon[29178]: mgrmap e7: x(active, since 1.00591s) 2024-04-23T18:26:28.761 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:26:28.769 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-23T18:26:28.769 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T18:26:28.769 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.648+0000 7f596294d700 1 Processor -- start 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.648+0000 7f596294d700 1 -- start start 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.648+0000 7f596294d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 0x7f595c10b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.648+0000 7f596294d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f595c07be50 con 0x7f595c108ee0 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.649+0000 7f595bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 0x7f595c10b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.649+0000 7f595bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 0x7f595c10b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57522/0 (socket says 172.21.15.132:57522) 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.649+0000 7f595bfff700 1 -- 172.21.15.132:0/642756596 learned_addr learned my addr 172.21.15.132:0/642756596 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.649+0000 7f595bfff700 1 -- 172.21.15.132:0/642756596 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c10b810 con 0x7f595c108ee0 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.649+0000 7f595bfff700 1 --2- 172.21.15.132:0/642756596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 0x7f595c10b2d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f59440098a0 tx=0x7f5944009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20de59ccc83a3fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.649+0000 7f595affd700 1 -- 172.21.15.132:0/642756596 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5944004160 con 0x7f595c108ee0 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.649+0000 7f595affd700 1 -- 172.21.15.132:0/642756596 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5944004300 con 0x7f595c108ee0 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.649+0000 7f595affd700 1 -- 172.21.15.132:0/642756596 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5944011af0 con 0x7f595c108ee0 2024-04-23T18:26:28.770 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.650+0000 7f596294d700 1 -- 172.21.15.132:0/642756596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 msgr2=0x7f595c10b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.650+0000 7f596294d700 1 --2- 172.21.15.132:0/642756596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 0x7f595c10b2d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f59440098a0 tx=0x7f5944009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.650+0000 7f596294d700 1 -- 172.21.15.132:0/642756596 shutdown_connections 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.650+0000 7f596294d700 1 --2- 172.21.15.132:0/642756596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 0x7f595c10b2d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.650+0000 7f596294d700 1 -- 172.21.15.132:0/642756596 >> 172.21.15.132:0/642756596 conn(0x7f595c102890 msgr2=0x7f595c104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.650+0000 7f596294d700 1 -- 172.21.15.132:0/642756596 shutdown_connections 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.650+0000 7f596294d700 1 -- 172.21.15.132:0/642756596 wait complete. 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.651+0000 7f596294d700 1 Processor -- start 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.651+0000 7f596294d700 1 -- start start 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.651+0000 7f596294d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 0x7f595c108760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.651+0000 7f596294d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f595c10c770 con 0x7f595c108ee0 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.651+0000 7f595bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 0x7f595c108760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.651+0000 7f595bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 0x7f595c108760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57526/0 (socket says 172.21.15.132:57526) 2024-04-23T18:26:28.771 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.651+0000 7f595bfff700 1 -- 172.21.15.132:0/2121415736 learned_addr learned my addr 172.21.15.132:0/2121415736 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.652+0000 7f595bfff700 1 -- 172.21.15.132:0/2121415736 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c108ca0 con 0x7f595c108ee0 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.652+0000 7f595bfff700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 0x7f595c108760 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5944009fb0 tx=0x7f5944011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.652+0000 7f59597fa700 1 -- 172.21.15.132:0/2121415736 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59440036a0 con 0x7f595c108ee0 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.652+0000 7f59597fa700 1 -- 172.21.15.132:0/2121415736 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f594401a430 con 0x7f595c108ee0 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.652+0000 7f596294d700 1 -- 172.21.15.132:0/2121415736 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595c106e90 con 0x7f595c108ee0 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.652+0000 7f59597fa700 1 -- 172.21.15.132:0/2121415736 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5944019550 con 0x7f595c108ee0 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.652+0000 7f596294d700 1 -- 172.21.15.132:0/2121415736 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595c1073d0 con 0x7f595c108ee0 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.653+0000 7f59597fa700 1 -- 172.21.15.132:0/2121415736 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f5944019850 con 0x7f595c108ee0 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.653+0000 7f59597fa700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 0x7f594803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.653+0000 7f596294d700 1 -- 172.21.15.132:0/2121415736 --> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f595c069b80 con 0x7f5948039560 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.653+0000 7f59597fa700 1 -- 172.21.15.132:0/2121415736 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f594404e8d0 con 0x7f595c108ee0 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.653+0000 7f595b7fe700 1 -- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 msgr2=0x7f594803ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/2811136616 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.653+0000 7f595b7fe700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 0x7f594803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.854+0000 7f595b7fe700 1 -- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 msgr2=0x7f594803ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/2811136616 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:22.854+0000 7f595b7fe700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 0x7f594803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:23.255+0000 7f595b7fe700 1 -- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 msgr2=0x7f594803ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/2811136616 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:23.255+0000 7f595b7fe700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 0x7f594803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:24.055+0000 7f595b7fe700 1 -- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 msgr2=0x7f594803ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/2811136616 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:24.055+0000 7f595b7fe700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 0x7f594803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:25.656+0000 7f595b7fe700 1 -- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 msgr2=0x7f594803ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/2811136616 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:25.656+0000 7f595b7fe700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 0x7f594803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:27.197+0000 7f59597fa700 1 -- 172.21.15.132:0/2121415736 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f594402d960 con 0x7f595c108ee0 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:27.197+0000 7f59597fa700 1 -- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 msgr2=0x7f594803ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:27.197+0000 7f59597fa700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 0x7f594803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.200+0000 7f59597fa700 1 -- 172.21.15.132:0/2121415736 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f5944018350 con 0x7f595c108ee0 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.200+0000 7f59597fa700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f594803cc60 0x7f594803f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.200+0000 7f59597fa700 1 -- 172.21.15.132:0/2121415736 --> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f594404ce60 con 0x7f594803cc60 2024-04-23T18:26:28.773 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.200+0000 7f595b7fe700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f594803cc60 0x7f594803f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.201+0000 7f595b7fe700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f594803cc60 0x7f594803f050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f594c003a10 tx=0x7f594c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.201+0000 7f59597fa700 1 -- 172.21.15.132:0/2121415736 <== mgr.14120 v2:172.21.15.132:6800/532999776 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f594404ce60 con 0x7f594803cc60 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.208+0000 7f596294d700 1 -- 172.21.15.132:0/2121415736 --> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f595c103b50 con 0x7f594803cc60 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.208+0000 7f59597fa700 1 -- 172.21.15.132:0/2121415736 <== mgr.14120 v2:172.21.15.132:6800/532999776 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f595c103b50 con 0x7f594803cc60 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.208+0000 7f596294d700 1 -- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f594803cc60 msgr2=0x7f594803f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.208+0000 7f596294d700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f594803cc60 0x7f594803f050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f594c003a10 tx=0x7f594c009290 comp rx=0 tx=0).stop 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.208+0000 7f596294d700 1 -- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 msgr2=0x7f595c108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.208+0000 7f596294d700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 0x7f595c108760 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5944009fb0 tx=0x7f5944011940 comp rx=0 tx=0).stop 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.208+0000 7f596294d700 1 -- 172.21.15.132:0/2121415736 shutdown_connections 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.208+0000 7f596294d700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f594803cc60 0x7f594803f050 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.208+0000 7f596294d700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c108ee0 0x7f595c108760 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.208+0000 7f596294d700 1 --2- 172.21.15.132:0/2121415736 >> [v2:172.21.15.132:6800/2811136616,v1:172.21.15.132:6801/2811136616] conn(0x7f5948039560 0x7f594803ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.208+0000 7f596294d700 1 -- 172.21.15.132:0/2121415736 >> 172.21.15.132:0/2121415736 conn(0x7f595c102890 msgr2=0x7f595c103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.209+0000 7f596294d700 1 -- 172.21.15.132:0/2121415736 shutdown_connections 2024-04-23T18:26:28.775 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:28.209+0000 7f596294d700 1 -- 172.21.15.132:0/2121415736 wait complete. 2024-04-23T18:26:28.775 INFO:teuthology.orchestra.run.smithi132.stdout:mgr epoch 5 is available 2024-04-23T18:26:28.775 INFO:teuthology.orchestra.run.smithi132.stdout:Setting orchestrator backend to cephadm... 2024-04-23T18:26:29.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:29 smithi132 ceph-mon[29178]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T18:26:29.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:29 smithi132 ceph-mon[29178]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T18:26:29.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:29 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.194+0000 7f0752ef5700 1 Processor -- start 2024-04-23T18:26:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.194+0000 7f0752ef5700 1 -- start start 2024-04-23T18:26:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.194+0000 7f0752ef5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 0x7f074c108840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.194+0000 7f0752ef5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f074c108e10 con 0x7f074c108420 2024-04-23T18:26:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.195+0000 7f0750c91700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 0x7f074c108840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.195+0000 7f0750c91700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 0x7f074c108840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57600/0 (socket says 172.21.15.132:57600) 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.195+0000 7f0750c91700 1 -- 172.21.15.132:0/809197587 learned_addr learned my addr 172.21.15.132:0/809197587 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.195+0000 7f0750c91700 1 -- 172.21.15.132:0/809197587 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074c1095d0 con 0x7f074c108420 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.195+0000 7f0750c91700 1 --2- 172.21.15.132:0/809197587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 0x7f074c108840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f07400098a0 tx=0x7f0740009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f794594ffa999137 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.195+0000 7f074b7fe700 1 -- 172.21.15.132:0/809197587 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0740004160 con 0x7f074c108420 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.195+0000 7f074b7fe700 1 -- 172.21.15.132:0/809197587 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0740004300 con 0x7f074c108420 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.195+0000 7f074b7fe700 1 -- 172.21.15.132:0/809197587 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0740011af0 con 0x7f074c108420 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.196+0000 7f0752ef5700 1 -- 172.21.15.132:0/809197587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 msgr2=0x7f074c108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.196+0000 7f0752ef5700 1 --2- 172.21.15.132:0/809197587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 0x7f074c108840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f07400098a0 tx=0x7f0740009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.196+0000 7f0752ef5700 1 -- 172.21.15.132:0/809197587 shutdown_connections 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.196+0000 7f0752ef5700 1 --2- 172.21.15.132:0/809197587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 0x7f074c108840 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.196+0000 7f0752ef5700 1 -- 172.21.15.132:0/809197587 >> 172.21.15.132:0/809197587 conn(0x7f074c103a20 msgr2=0x7f074c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.196+0000 7f0752ef5700 1 -- 172.21.15.132:0/809197587 shutdown_connections 2024-04-23T18:26:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.196+0000 7f0752ef5700 1 -- 172.21.15.132:0/809197587 wait complete. 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.196+0000 7f0752ef5700 1 Processor -- start 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.197+0000 7f0752ef5700 1 -- start start 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.197+0000 7f0752ef5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 0x7f074c19c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.197+0000 7f0752ef5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f074c10c980 con 0x7f074c108420 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.197+0000 7f0750c91700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 0x7f074c19c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.197+0000 7f0750c91700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 0x7f074c19c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57606/0 (socket says 172.21.15.132:57606) 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.197+0000 7f0750c91700 1 -- 172.21.15.132:0/3530743441 learned_addr learned my addr 172.21.15.132:0/3530743441 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.197+0000 7f0750c91700 1 -- 172.21.15.132:0/3530743441 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074c19c8c0 con 0x7f074c108420 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.197+0000 7f0750c91700 1 --2- 172.21.15.132:0/3530743441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 0x7f074c19c380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0740003f40 tx=0x7f0740011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.197+0000 7f0749ffb700 1 -- 172.21.15.132:0/3530743441 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07400038b0 con 0x7f074c108420 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.197+0000 7f0749ffb700 1 -- 172.21.15.132:0/3530743441 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0740003a50 con 0x7f074c108420 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.197+0000 7f0752ef5700 1 -- 172.21.15.132:0/3530743441 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074c19cb50 con 0x7f074c108420 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.197+0000 7f0749ffb700 1 -- 172.21.15.132:0/3530743441 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f074001a5c0 con 0x7f074c108420 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.198+0000 7f0752ef5700 1 -- 172.21.15.132:0/3530743441 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f074c19f840 con 0x7f074c108420 2024-04-23T18:26:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.198+0000 7f0749ffb700 1 -- 172.21.15.132:0/3530743441 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f074001a8c0 con 0x7f074c108420 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.198+0000 7f0749ffb700 1 --2- 172.21.15.132:0/3530743441 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f073c039450 0x7f073c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.198+0000 7f0749ffb700 1 -- 172.21.15.132:0/3530743441 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f074004ec10 con 0x7f074c108420 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.199+0000 7f074bfff700 1 --2- 172.21.15.132:0/3530743441 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f073c039450 0x7f073c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.199+0000 7f0752ef5700 1 -- 172.21.15.132:0/3530743441 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f074c0640a0 con 0x7f074c108420 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.199+0000 7f074bfff700 1 --2- 172.21.15.132:0/3530743441 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f073c039450 0x7f073c03b910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0738006fd0 tx=0x7f0738006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.204+0000 7f0749ffb700 1 -- 172.21.15.132:0/3530743441 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f074000b3e0 con 0x7f074c108420 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.251+0000 7f0749ffb700 1 -- 172.21.15.132:0/3530743441 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f074002da00 con 0x7f074c108420 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.421+0000 7f0752ef5700 1 -- 172.21.15.132:0/3530743441 --> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f074c104ce0 con 0x7f073c039450 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.426+0000 7f0749ffb700 1 -- 172.21.15.132:0/3530743441 <== mgr.14120 v2:172.21.15.132:6800/532999776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f074c104ce0 con 0x7f073c039450 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.428+0000 7f0752ef5700 1 -- 172.21.15.132:0/3530743441 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f073c039450 msgr2=0x7f073c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.428+0000 7f0752ef5700 1 --2- 172.21.15.132:0/3530743441 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f073c039450 0x7f073c03b910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0738006fd0 tx=0x7f0738006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.428+0000 7f0752ef5700 1 -- 172.21.15.132:0/3530743441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 msgr2=0x7f074c19c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.428+0000 7f0752ef5700 1 --2- 172.21.15.132:0/3530743441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 0x7f074c19c380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0740003f40 tx=0x7f0740011890 comp rx=0 tx=0).stop 2024-04-23T18:26:30.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.428+0000 7f0752ef5700 1 -- 172.21.15.132:0/3530743441 shutdown_connections 2024-04-23T18:26:30.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.428+0000 7f0752ef5700 1 --2- 172.21.15.132:0/3530743441 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f073c039450 0x7f073c03b910 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:30.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.428+0000 7f0752ef5700 1 --2- 172.21.15.132:0/3530743441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f074c108420 0x7f074c19c380 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:30.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.428+0000 7f0752ef5700 1 -- 172.21.15.132:0/3530743441 >> 172.21.15.132:0/3530743441 conn(0x7f074c103a20 msgr2=0x7f074c1045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:30.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.428+0000 7f0752ef5700 1 -- 172.21.15.132:0/3530743441 shutdown_connections 2024-04-23T18:26:30.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:29.428+0000 7f0752ef5700 1 -- 172.21.15.132:0/3530743441 wait complete. 2024-04-23T18:26:30.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:30 smithi132 ceph-mon[29178]: [23/Apr/2024:18:26:28] ENGINE Bus STARTING 2024-04-23T18:26:30.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:30 smithi132 ceph-mon[29178]: [23/Apr/2024:18:26:28] ENGINE Serving on https://172.21.15.132:7150 2024-04-23T18:26:30.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:30 smithi132 ceph-mon[29178]: [23/Apr/2024:18:26:28] ENGINE Serving on http://172.21.15.132:8765 2024-04-23T18:26:30.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:30 smithi132 ceph-mon[29178]: [23/Apr/2024:18:26:28] ENGINE Bus STARTED 2024-04-23T18:26:30.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:30 smithi132 ceph-mon[29178]: mgrmap e8: x(active, since 2s) 2024-04-23T18:26:30.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:30 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:30.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:30 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:31.254 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-23T18:26:31.254 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.457+0000 7fe9979fc700 1 Processor -- start 2024-04-23T18:26:31.254 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.457+0000 7fe9979fc700 1 -- start start 2024-04-23T18:26:31.254 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.457+0000 7fe9979fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 0x7fe99010b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.254 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.457+0000 7fe9979fc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe99007beb0 con 0x7fe990108e80 2024-04-23T18:26:31.254 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.457+0000 7fe995798700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 0x7fe99010b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.254 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.457+0000 7fe995798700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 0x7fe99010b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60914/0 (socket says 172.21.15.132:60914) 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.457+0000 7fe995798700 1 -- 172.21.15.132:0/3584183584 learned_addr learned my addr 172.21.15.132:0/3584183584 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.458+0000 7fe995798700 1 -- 172.21.15.132:0/3584183584 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99010b7b0 con 0x7fe990108e80 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.458+0000 7fe995798700 1 --2- 172.21.15.132:0/3584183584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 0x7fe99010b270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9840098a0 tx=0x7fe984009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9577584b4a4a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.458+0000 7fe98ffff700 1 -- 172.21.15.132:0/3584183584 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe984004160 con 0x7fe990108e80 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.458+0000 7fe98ffff700 1 -- 172.21.15.132:0/3584183584 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe984004300 con 0x7fe990108e80 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.458+0000 7fe98ffff700 1 -- 172.21.15.132:0/3584183584 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe984011af0 con 0x7fe990108e80 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.458+0000 7fe9979fc700 1 -- 172.21.15.132:0/3584183584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 msgr2=0x7fe99010b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.458+0000 7fe9979fc700 1 --2- 172.21.15.132:0/3584183584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 0x7fe99010b270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9840098a0 tx=0x7fe984009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.459+0000 7fe9979fc700 1 -- 172.21.15.132:0/3584183584 shutdown_connections 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.459+0000 7fe9979fc700 1 --2- 172.21.15.132:0/3584183584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 0x7fe99010b270 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.459+0000 7fe9979fc700 1 -- 172.21.15.132:0/3584183584 >> 172.21.15.132:0/3584183584 conn(0x7fe990102890 msgr2=0x7fe990104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.459+0000 7fe9979fc700 1 -- 172.21.15.132:0/3584183584 shutdown_connections 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.459+0000 7fe9979fc700 1 -- 172.21.15.132:0/3584183584 wait complete. 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.459+0000 7fe9979fc700 1 Processor -- start 2024-04-23T18:26:31.255 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.459+0000 7fe9979fc700 1 -- start start 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.459+0000 7fe9979fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 0x7fe990108700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.459+0000 7fe9979fc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe99010c700 con 0x7fe990108e80 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.460+0000 7fe995798700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 0x7fe990108700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.460+0000 7fe995798700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 0x7fe990108700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60920/0 (socket says 172.21.15.132:60920) 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.460+0000 7fe995798700 1 -- 172.21.15.132:0/3079961865 learned_addr learned my addr 172.21.15.132:0/3079961865 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.460+0000 7fe995798700 1 -- 172.21.15.132:0/3079961865 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe990108c40 con 0x7fe990108e80 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.460+0000 7fe995798700 1 --2- 172.21.15.132:0/3079961865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 0x7fe990108700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe984009fb0 tx=0x7fe984011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.460+0000 7fe98e7fc700 1 -- 172.21.15.132:0/3079961865 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9840036a0 con 0x7fe990108e80 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.460+0000 7fe98e7fc700 1 -- 172.21.15.132:0/3079961865 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe98401a430 con 0x7fe990108e80 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.460+0000 7fe98e7fc700 1 -- 172.21.15.132:0/3079961865 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe984019550 con 0x7fe990108e80 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.460+0000 7fe9979fc700 1 -- 172.21.15.132:0/3079961865 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe990106e30 con 0x7fe990108e80 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.460+0000 7fe9979fc700 1 -- 172.21.15.132:0/3079961865 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe990107370 con 0x7fe990108e80 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.461+0000 7fe98e7fc700 1 -- 172.21.15.132:0/3079961865 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe984003840 con 0x7fe990108e80 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.461+0000 7fe9979fc700 1 -- 172.21.15.132:0/3079961865 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe990069b80 con 0x7fe990108e80 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.461+0000 7fe98e7fc700 1 --2- 172.21.15.132:0/3079961865 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fe9800395b0 0x7fe98003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.256 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.461+0000 7fe98e7fc700 1 -- 172.21.15.132:0/3079961865 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe98404d860 con 0x7fe990108e80 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.461+0000 7fe994f97700 1 --2- 172.21.15.132:0/3079961865 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fe9800395b0 0x7fe98003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.462+0000 7fe994f97700 1 --2- 172.21.15.132:0/3079961865 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fe9800395b0 0x7fe98003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe97c006fd0 tx=0x7fe97c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.466+0000 7fe98e7fc700 1 -- 172.21.15.132:0/3079961865 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe98400b3e0 con 0x7fe990108e80 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.665+0000 7fe9979fc700 1 -- 172.21.15.132:0/3079961865 --> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fe990103b70 con 0x7fe9800395b0 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.665+0000 7fe98e7fc700 1 -- 172.21.15.132:0/3079961865 <== mgr.14120 v2:172.21.15.132:6800/532999776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fe990103b70 con 0x7fe9800395b0 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.667+0000 7fe9979fc700 1 -- 172.21.15.132:0/3079961865 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fe9800395b0 msgr2=0x7fe98003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.667+0000 7fe9979fc700 1 --2- 172.21.15.132:0/3079961865 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fe9800395b0 0x7fe98003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe97c006fd0 tx=0x7fe97c006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.667+0000 7fe9979fc700 1 -- 172.21.15.132:0/3079961865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 msgr2=0x7fe990108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.667+0000 7fe9979fc700 1 --2- 172.21.15.132:0/3079961865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 0x7fe990108700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe984009fb0 tx=0x7fe984011940 comp rx=0 tx=0).stop 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.667+0000 7fe9979fc700 1 -- 172.21.15.132:0/3079961865 shutdown_connections 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.667+0000 7fe9979fc700 1 --2- 172.21.15.132:0/3079961865 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fe9800395b0 0x7fe98003ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.667+0000 7fe9979fc700 1 --2- 172.21.15.132:0/3079961865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe990108e80 0x7fe990108700 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.667+0000 7fe9979fc700 1 -- 172.21.15.132:0/3079961865 >> 172.21.15.132:0/3079961865 conn(0x7fe990102890 msgr2=0x7fe990103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.668+0000 7fe9979fc700 1 -- 172.21.15.132:0/3079961865 shutdown_connections 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:30.668+0000 7fe9979fc700 1 -- 172.21.15.132:0/3079961865 wait complete. 2024-04-23T18:26:31.257 INFO:teuthology.orchestra.run.smithi132.stdout:Generating ssh key... 2024-04-23T18:26:31.599 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:31 smithi132 ceph-mon[29178]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:32.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-mon[29178]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:32.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:32.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: Generating public/private rsa key pair. 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: Your identification has been saved in /tmp/tmpqxowa6xp/key. 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: Your public key has been saved in /tmp/tmpqxowa6xp/key.pub. 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: The key fingerprint is: 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: SHA256:c6ly7ePZ5XxakZ6XeK075XqqPUuSCxYxXAZRbEqmm80 ceph-c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: The key's randomart image is: 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: +---[RSA 3072]----+ 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: | o=+ | 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: | .ooo | 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: | ++o | 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: | . .+ .| 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: | S=+ o | 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: | o=E. .o *| 2024-04-23T18:26:32.613 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: | . o + o.oB+| 2024-04-23T18:26:32.614 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: | o o.+ O+o=| 2024-04-23T18:26:32.614 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: | .+.+.@@ | 2024-04-23T18:26:32.614 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: +----[SHA256]-----+ 2024-04-23T18:26:32.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.704+0000 7fadf7b7f700 1 Processor -- start 2024-04-23T18:26:32.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.704+0000 7fadf7b7f700 1 -- start start 2024-04-23T18:26:32.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.704+0000 7fadf7b7f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 0x7fadf007c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:32.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.704+0000 7fadf7b7f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fadf007c910 con 0x7fadf007de90 2024-04-23T18:26:32.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.704+0000 7fadf591b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 0x7fadf007c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:32.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.705+0000 7fadf591b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 0x7fadf007c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60932/0 (socket says 172.21.15.132:60932) 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.705+0000 7fadf591b700 1 -- 172.21.15.132:0/1989500807 learned_addr learned my addr 172.21.15.132:0/1989500807 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.705+0000 7fadf591b700 1 -- 172.21.15.132:0/1989500807 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf007ca90 con 0x7fadf007de90 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.705+0000 7fadf591b700 1 --2- 172.21.15.132:0/1989500807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 0x7fadf007c340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fade0009e80 tx=0x7fade000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d785dd91cdb08a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.705+0000 7fadf4919700 1 -- 172.21.15.132:0/1989500807 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade0017070 con 0x7fadf007de90 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.705+0000 7fadf4919700 1 -- 172.21.15.132:0/1989500807 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fade000b8c0 con 0x7fadf007de90 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.705+0000 7fadf4919700 1 -- 172.21.15.132:0/1989500807 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade0004490 con 0x7fadf007de90 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.706+0000 7fadf7b7f700 1 -- 172.21.15.132:0/1989500807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 msgr2=0x7fadf007c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.706+0000 7fadf7b7f700 1 --2- 172.21.15.132:0/1989500807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 0x7fadf007c340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fade0009e80 tx=0x7fade000b330 comp rx=0 tx=0).stop 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.706+0000 7fadf7b7f700 1 -- 172.21.15.132:0/1989500807 shutdown_connections 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.706+0000 7fadf7b7f700 1 --2- 172.21.15.132:0/1989500807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 0x7fadf007c340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.706+0000 7fadf7b7f700 1 -- 172.21.15.132:0/1989500807 >> 172.21.15.132:0/1989500807 conn(0x7fadf0103a80 msgr2=0x7fadf0105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.706+0000 7fadf7b7f700 1 -- 172.21.15.132:0/1989500807 shutdown_connections 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.706+0000 7fadf7b7f700 1 -- 172.21.15.132:0/1989500807 wait complete. 2024-04-23T18:26:32.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.706+0000 7fadf7b7f700 1 Processor -- start 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.706+0000 7fadf7b7f700 1 -- start start 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.707+0000 7fadf7b7f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 0x7fadf019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.707+0000 7fadf7b7f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fadf010e990 con 0x7fadf007de90 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.707+0000 7fadf591b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 0x7fadf019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.707+0000 7fadf591b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 0x7fadf019e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60934/0 (socket says 172.21.15.132:60934) 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.707+0000 7fadf591b700 1 -- 172.21.15.132:0/3655101154 learned_addr learned my addr 172.21.15.132:0/3655101154 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.707+0000 7fadf591b700 1 -- 172.21.15.132:0/3655101154 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf019ea90 con 0x7fadf007de90 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.707+0000 7fadf591b700 1 --2- 172.21.15.132:0/3655101154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 0x7fadf019e550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fade000b7b0 tx=0x7fade000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.707+0000 7fadeeffd700 1 -- 172.21.15.132:0/3655101154 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade0020020 con 0x7fadf007de90 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.707+0000 7fadf7b7f700 1 -- 172.21.15.132:0/3655101154 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf019ed20 con 0x7fadf007de90 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.707+0000 7fadeeffd700 1 -- 172.21.15.132:0/3655101154 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fade0003d10 con 0x7fadf007de90 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.707+0000 7fadeeffd700 1 -- 172.21.15.132:0/3655101154 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade0017040 con 0x7fadf007de90 2024-04-23T18:26:32.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.707+0000 7fadf7b7f700 1 -- 172.21.15.132:0/3655101154 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf01a1a10 con 0x7fadf007de90 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.708+0000 7fadeeffd700 1 -- 172.21.15.132:0/3655101154 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fade0023020 con 0x7fadf007de90 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.708+0000 7fadeeffd700 1 --2- 172.21.15.132:0/3655101154 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fadd80395b0 0x7fadd803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.708+0000 7fadeeffd700 1 -- 172.21.15.132:0/3655101154 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fade000f050 con 0x7fadf007de90 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.708+0000 7fadf7b7f700 1 -- 172.21.15.132:0/3655101154 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf00640a0 con 0x7fadf007de90 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.708+0000 7fadf511a700 1 --2- 172.21.15.132:0/3655101154 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fadd80395b0 0x7fadd803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.709+0000 7fadf511a700 1 --2- 172.21.15.132:0/3655101154 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fadd80395b0 0x7fadd803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fade4006fd0 tx=0x7fade4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.713+0000 7fadeeffd700 1 -- 172.21.15.132:0/3655101154 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fade002a020 con 0x7fadf007de90 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:31.914+0000 7fadf7b7f700 1 -- 172.21.15.132:0/3655101154 --> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fadf0104ca0 con 0x7fadd80395b0 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:32.200+0000 7fadeeffd700 1 -- 172.21.15.132:0/3655101154 <== mgr.14120 v2:172.21.15.132:6800/532999776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fadf0104ca0 con 0x7fadd80395b0 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:32.202+0000 7fadf7b7f700 1 -- 172.21.15.132:0/3655101154 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fadd80395b0 msgr2=0x7fadd803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:32.202+0000 7fadf7b7f700 1 --2- 172.21.15.132:0/3655101154 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fadd80395b0 0x7fadd803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fade4006fd0 tx=0x7fade4006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:32.202+0000 7fadf7b7f700 1 -- 172.21.15.132:0/3655101154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 msgr2=0x7fadf019e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:32.202+0000 7fadf7b7f700 1 --2- 172.21.15.132:0/3655101154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 0x7fadf019e550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fade000b7b0 tx=0x7fade000be90 comp rx=0 tx=0).stop 2024-04-23T18:26:32.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:32.202+0000 7fadf7b7f700 1 -- 172.21.15.132:0/3655101154 shutdown_connections 2024-04-23T18:26:32.660 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:32.202+0000 7fadf7b7f700 1 --2- 172.21.15.132:0/3655101154 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fadd80395b0 0x7fadd803ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.660 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:32.202+0000 7fadf7b7f700 1 --2- 172.21.15.132:0/3655101154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadf007de90 0x7fadf019e550 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.660 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:32.202+0000 7fadf7b7f700 1 -- 172.21.15.132:0/3655101154 >> 172.21.15.132:0/3655101154 conn(0x7fadf0103a80 msgr2=0x7fadf0104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:32.660 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:32.202+0000 7fadf7b7f700 1 -- 172.21.15.132:0/3655101154 shutdown_connections 2024-04-23T18:26:32.660 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:32.202+0000 7fadf7b7f700 1 -- 172.21.15.132:0/3655101154 wait complete. 2024-04-23T18:26:33.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:33 smithi132 ceph-mon[29178]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:33.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:33 smithi132 ceph-mon[29178]: Generating ssh key... 2024-04-23T18:26:33.859 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHj4S72TxmaMvOqsIsUwYoMPFCypC7LiLyuAImJ3Idx1lHq0LsHVkjdKTxfM34kAmuDE6NxkcnKsbmqcpKnQB2cphZXRFL5zf6s/5gPpU8tqRaqrWVJsJHiFoSSp/ZJMigqbOQBqAQhkzewh8JFpgOr91UZKaWHvcQbojaoeJeEX7n6w9FbuDnn0OvU5lFoGiw9ElDuwbvyI184rA+TmBD8kDIDDmKfzKXf8TZuXt5RWOCUM4N4IUyjcZLXmDx4tE/0SEMAqXq/gNZuxM2DH2kIvEQOmAkfApK0H3C2hZ76RQ+kZ4z1j6PdKlWr2HgB5LSce/PBcr+2V3sdCtGjB1JeqcHrsWyZELl8TJkn7sEr5bsesO8YsRERT0qmlK+DzfTL7BWKDjcwlSP2PaEZ8LZL7Fn3FpFSE0buli5c0WPLpFfvXyi7zQaESt6+DPriDHXmERYA6a9HChfTum0FdZKOkRKBiJnYfQQEjDV6VqqwgMSplpgYdRIjvbZ+/lalVk= ceph-c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:26:33.859 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.086+0000 7fc75259e700 1 Processor -- start 2024-04-23T18:26:33.859 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.086+0000 7fc75259e700 1 -- start start 2024-04-23T18:26:33.859 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.086+0000 7fc75259e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 0x7fc74c10b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.859 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.086+0000 7fc75259e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c07beb0 con 0x7fc74c108ef0 2024-04-23T18:26:33.859 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.086+0000 7fc74bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 0x7fc74c10b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.086+0000 7fc74bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 0x7fc74c10b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60942/0 (socket says 172.21.15.132:60942) 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.086+0000 7fc74bfff700 1 -- 172.21.15.132:0/1761982041 learned_addr learned my addr 172.21.15.132:0/1761982041 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.087+0000 7fc74bfff700 1 -- 172.21.15.132:0/1761982041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c10b820 con 0x7fc74c108ef0 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.087+0000 7fc74bfff700 1 --2- 172.21.15.132:0/1761982041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 0x7fc74c10b2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0098a0 tx=0x7fc73c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fab1fd818c51b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.087+0000 7fc74affd700 1 -- 172.21.15.132:0/1761982041 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc73c004160 con 0x7fc74c108ef0 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.087+0000 7fc74affd700 1 -- 172.21.15.132:0/1761982041 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc73c004300 con 0x7fc74c108ef0 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.087+0000 7fc74affd700 1 -- 172.21.15.132:0/1761982041 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc73c011af0 con 0x7fc74c108ef0 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.088+0000 7fc75259e700 1 -- 172.21.15.132:0/1761982041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 msgr2=0x7fc74c10b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.088+0000 7fc75259e700 1 --2- 172.21.15.132:0/1761982041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 0x7fc74c10b2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0098a0 tx=0x7fc73c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.088+0000 7fc75259e700 1 -- 172.21.15.132:0/1761982041 shutdown_connections 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.088+0000 7fc75259e700 1 --2- 172.21.15.132:0/1761982041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 0x7fc74c10b2e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.088+0000 7fc75259e700 1 -- 172.21.15.132:0/1761982041 >> 172.21.15.132:0/1761982041 conn(0x7fc74c102940 msgr2=0x7fc74c104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.088+0000 7fc75259e700 1 -- 172.21.15.132:0/1761982041 shutdown_connections 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.088+0000 7fc75259e700 1 -- 172.21.15.132:0/1761982041 wait complete. 2024-04-23T18:26:33.860 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.088+0000 7fc75259e700 1 Processor -- start 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.088+0000 7fc75259e700 1 -- start start 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.088+0000 7fc75259e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 0x7fc74c19c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.088+0000 7fc75259e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c10c770 con 0x7fc74c108ef0 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.089+0000 7fc74bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 0x7fc74c19c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.089+0000 7fc74bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 0x7fc74c19c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60944/0 (socket says 172.21.15.132:60944) 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.089+0000 7fc74bfff700 1 -- 172.21.15.132:0/554126619 learned_addr learned my addr 172.21.15.132:0/554126619 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.089+0000 7fc74bfff700 1 -- 172.21.15.132:0/554126619 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c19c8b0 con 0x7fc74c108ef0 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.089+0000 7fc74bfff700 1 --2- 172.21.15.132:0/554126619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 0x7fc74c19c370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc73c011ac0 tx=0x7fc73c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.089+0000 7fc7497fa700 1 -- 172.21.15.132:0/554126619 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc73c003970 con 0x7fc74c108ef0 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.089+0000 7fc7497fa700 1 -- 172.21.15.132:0/554126619 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc73c003b10 con 0x7fc74c108ef0 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.089+0000 7fc75259e700 1 -- 172.21.15.132:0/554126619 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74c19cba0 con 0x7fc74c108ef0 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.089+0000 7fc7497fa700 1 -- 172.21.15.132:0/554126619 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc73c024610 con 0x7fc74c108ef0 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.089+0000 7fc75259e700 1 -- 172.21.15.132:0/554126619 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc74c19f830 con 0x7fc74c108ef0 2024-04-23T18:26:33.861 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.090+0000 7fc7497fa700 1 -- 172.21.15.132:0/554126619 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc73c024840 con 0x7fc74c108ef0 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.090+0000 7fc75259e700 1 -- 172.21.15.132:0/554126619 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc74c0640a0 con 0x7fc74c108ef0 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.090+0000 7fc7497fa700 1 --2- 172.21.15.132:0/554126619 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fc734039600 0x7fc73403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.090+0000 7fc7497fa700 1 -- 172.21.15.132:0/554126619 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc73c04d9a0 con 0x7fc74c108ef0 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.090+0000 7fc74b7fe700 1 --2- 172.21.15.132:0/554126619 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fc734039600 0x7fc73403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.091+0000 7fc74b7fe700 1 --2- 172.21.15.132:0/554126619 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fc734039600 0x7fc73403bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc740006fd0 tx=0x7fc740006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.095+0000 7fc7497fa700 1 -- 172.21.15.132:0/554126619 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc73c02d910 con 0x7fc74c108ef0 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.292+0000 7fc75259e700 1 -- 172.21.15.132:0/554126619 --> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fc74c19fa00 con 0x7fc734039600 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.293+0000 7fc7497fa700 1 -- 172.21.15.132:0/554126619 <== mgr.14120 v2:172.21.15.132:6800/532999776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fc74c19fa00 con 0x7fc734039600 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.295+0000 7fc75259e700 1 -- 172.21.15.132:0/554126619 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fc734039600 msgr2=0x7fc73403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.295+0000 7fc75259e700 1 --2- 172.21.15.132:0/554126619 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fc734039600 0x7fc73403bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc740006fd0 tx=0x7fc740006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.295+0000 7fc75259e700 1 -- 172.21.15.132:0/554126619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 msgr2=0x7fc74c19c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.295+0000 7fc75259e700 1 --2- 172.21.15.132:0/554126619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 0x7fc74c19c370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc73c011ac0 tx=0x7fc73c011ba0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.295+0000 7fc75259e700 1 -- 172.21.15.132:0/554126619 shutdown_connections 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.295+0000 7fc75259e700 1 --2- 172.21.15.132:0/554126619 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fc734039600 0x7fc73403bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.862 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.295+0000 7fc75259e700 1 --2- 172.21.15.132:0/554126619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc74c108ef0 0x7fc74c19c370 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.863 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.295+0000 7fc75259e700 1 -- 172.21.15.132:0/554126619 >> 172.21.15.132:0/554126619 conn(0x7fc74c102940 msgr2=0x7fc74c1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.863 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.295+0000 7fc75259e700 1 -- 172.21.15.132:0/554126619 shutdown_connections 2024-04-23T18:26:33.863 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:33.295+0000 7fc75259e700 1 -- 172.21.15.132:0/554126619 wait complete. 2024-04-23T18:26:33.863 INFO:teuthology.orchestra.run.smithi132.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:26:33.863 INFO:teuthology.orchestra.run.smithi132.stdout:Adding key to root@localhost authorized_keys... 2024-04-23T18:26:33.863 INFO:teuthology.orchestra.run.smithi132.stdout:Adding host smithi132... 2024-04-23T18:26:34.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:34 smithi132 ceph-mon[29178]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:35 smithi132 ceph-mon[29178]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi132", "addr": "172.21.15.132", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:35 smithi132 ceph-mon[29178]: Deploying cephadm binary to smithi132 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout Added host 'smithi132' with addr '172.21.15.132' 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.277+0000 7f75362fe700 1 Processor -- start 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.277+0000 7f75362fe700 1 -- start start 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.277+0000 7f75362fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 0x7f75301088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.277+0000 7f75362fe700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7530108e90 con 0x7f75301084a0 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.278+0000 7f752ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 0x7f75301088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.278+0000 7f752ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 0x7f75301088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60948/0 (socket says 172.21.15.132:60948) 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.278+0000 7f752ffff700 1 -- 172.21.15.132:0/3446802397 learned_addr learned my addr 172.21.15.132:0/3446802397 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.278+0000 7f752ffff700 1 -- 172.21.15.132:0/3446802397 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7530109650 con 0x7f75301084a0 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.278+0000 7f752ffff700 1 --2- 172.21.15.132:0/3446802397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 0x7f75301088c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f75200098a0 tx=0x7f7520009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c1d036f5940899 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.278+0000 7f752effd700 1 -- 172.21.15.132:0/3446802397 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7520004160 con 0x7f75301084a0 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.278+0000 7f752effd700 1 -- 172.21.15.132:0/3446802397 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7520004300 con 0x7f75301084a0 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.278+0000 7f752effd700 1 -- 172.21.15.132:0/3446802397 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7520011af0 con 0x7f75301084a0 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.279+0000 7f75362fe700 1 -- 172.21.15.132:0/3446802397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 msgr2=0x7f75301088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.279+0000 7f75362fe700 1 --2- 172.21.15.132:0/3446802397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 0x7f75301088c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f75200098a0 tx=0x7f7520009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.279+0000 7f75362fe700 1 -- 172.21.15.132:0/3446802397 shutdown_connections 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.279+0000 7f75362fe700 1 --2- 172.21.15.132:0/3446802397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 0x7f75301088c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.279+0000 7f75362fe700 1 -- 172.21.15.132:0/3446802397 >> 172.21.15.132:0/3446802397 conn(0x7f7530103ac0 msgr2=0x7f7530105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.279+0000 7f75362fe700 1 -- 172.21.15.132:0/3446802397 shutdown_connections 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.279+0000 7f75362fe700 1 -- 172.21.15.132:0/3446802397 wait complete. 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.279+0000 7f75362fe700 1 Processor -- start 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.279+0000 7f75362fe700 1 -- start start 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.280+0000 7f75362fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 0x7f753019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.280+0000 7f75362fe700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f753010ca00 con 0x7f75301084a0 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.280+0000 7f752ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 0x7f753019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.280+0000 7f752ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 0x7f753019c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60952/0 (socket says 172.21.15.132:60952) 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.280+0000 7f752ffff700 1 -- 172.21.15.132:0/3041135964 learned_addr learned my addr 172.21.15.132:0/3041135964 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.280+0000 7f752ffff700 1 -- 172.21.15.132:0/3041135964 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753019c940 con 0x7f75301084a0 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.280+0000 7f752ffff700 1 --2- 172.21.15.132:0/3041135964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 0x7f753019c400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f75200117b0 tx=0x7f7520011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.280+0000 7f752d7fa700 1 -- 172.21.15.132:0/3041135964 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7520003af0 con 0x7f75301084a0 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.280+0000 7f752d7fa700 1 -- 172.21.15.132:0/3041135964 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7520020070 con 0x7f75301084a0 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.280+0000 7f75362fe700 1 -- 172.21.15.132:0/3041135964 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753019cbd0 con 0x7f75301084a0 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.280+0000 7f752d7fa700 1 -- 172.21.15.132:0/3041135964 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7520019630 con 0x7f75301084a0 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.280+0000 7f75362fe700 1 -- 172.21.15.132:0/3041135964 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753019f8c0 con 0x7f75301084a0 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.281+0000 7f752d7fa700 1 -- 172.21.15.132:0/3041135964 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f75200198a0 con 0x7f75301084a0 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.281+0000 7f75362fe700 1 -- 172.21.15.132:0/3041135964 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75300640a0 con 0x7f75301084a0 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.281+0000 7f752d7fa700 1 --2- 172.21.15.132:0/3041135964 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7518039600 0x7f751803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.281+0000 7f752d7fa700 1 -- 172.21.15.132:0/3041135964 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f752004dd40 con 0x7f75301084a0 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.282+0000 7f752f7fe700 1 --2- 172.21.15.132:0/3041135964 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7518039600 0x7f751803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.282+0000 7f752f7fe700 1 --2- 172.21.15.132:0/3041135964 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7518039600 0x7f751803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7524006fd0 tx=0x7f7524006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.286+0000 7f752d7fa700 1 -- 172.21.15.132:0/3041135964 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f752002bd00 con 0x7f75301084a0 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:34.494+0000 7f75362fe700 1 -- 172.21.15.132:0/3041135964 --> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi132", "addr": "172.21.15.132", "target": ["mon-mgr", ""]}) -- 0x7f7530104ce0 con 0x7f7518039600 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:38.603+0000 7f752d7fa700 1 -- 172.21.15.132:0/3041135964 <== mgr.14120 v2:172.21.15.132:6800/532999776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7530104ce0 con 0x7f7518039600 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:38.605+0000 7f75362fe700 1 -- 172.21.15.132:0/3041135964 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7518039600 msgr2=0x7f751803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:38.605+0000 7f75362fe700 1 --2- 172.21.15.132:0/3041135964 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7518039600 0x7f751803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7524006fd0 tx=0x7f7524006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:38.606+0000 7f75362fe700 1 -- 172.21.15.132:0/3041135964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 msgr2=0x7f753019c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:38.606+0000 7f75362fe700 1 --2- 172.21.15.132:0/3041135964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 0x7f753019c400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f75200117b0 tx=0x7f7520011b70 comp rx=0 tx=0).stop 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:38.606+0000 7f75362fe700 1 -- 172.21.15.132:0/3041135964 shutdown_connections 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:38.606+0000 7f75362fe700 1 --2- 172.21.15.132:0/3041135964 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7518039600 0x7f751803bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:38.606+0000 7f75362fe700 1 --2- 172.21.15.132:0/3041135964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75301084a0 0x7f753019c400 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:38.606+0000 7f75362fe700 1 -- 172.21.15.132:0/3041135964 >> 172.21.15.132:0/3041135964 conn(0x7f7530103ac0 msgr2=0x7f75301045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:38.606+0000 7f75362fe700 1 -- 172.21.15.132:0/3041135964 shutdown_connections 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:38.606+0000 7f75362fe700 1 -- 172.21.15.132:0/3041135964 wait complete. 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi132.stdout:Deploying unmanaged mon service... 2024-04-23T18:26:39.786 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:39 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:39.786 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:39 smithi132 ceph-mon[29178]: Added host smithi132 2024-04-23T18:26:39.786 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:39 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:40.500 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-23T18:26:40.500 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.545+0000 7f692a20d700 1 Processor -- start 2024-04-23T18:26:40.500 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.545+0000 7f692a20d700 1 -- start start 2024-04-23T18:26:40.500 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.545+0000 7f692a20d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 0x7f692407c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:40.500 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.545+0000 7f692a20d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f692407c910 con 0x7f692407de90 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.545+0000 7f69237fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 0x7f692407c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.545+0000 7f69237fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 0x7f692407c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36646/0 (socket says 172.21.15.132:36646) 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.545+0000 7f69237fe700 1 -- 172.21.15.132:0/3798463552 learned_addr learned my addr 172.21.15.132:0/3798463552 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.545+0000 7f69237fe700 1 -- 172.21.15.132:0/3798463552 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692407ca90 con 0x7f692407de90 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.546+0000 7f69237fe700 1 --2- 172.21.15.132:0/3798463552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 0x7f692407c340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f69140098a0 tx=0x7f6914009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560d73fc423f1b5b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.546+0000 7f69227fc700 1 -- 172.21.15.132:0/3798463552 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6914004160 con 0x7f692407de90 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.546+0000 7f69227fc700 1 -- 172.21.15.132:0/3798463552 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6914004300 con 0x7f692407de90 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.546+0000 7f69227fc700 1 -- 172.21.15.132:0/3798463552 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6914011af0 con 0x7f692407de90 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.546+0000 7f692a20d700 1 -- 172.21.15.132:0/3798463552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 msgr2=0x7f692407c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.546+0000 7f692a20d700 1 --2- 172.21.15.132:0/3798463552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 0x7f692407c340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f69140098a0 tx=0x7f6914009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.546+0000 7f692a20d700 1 -- 172.21.15.132:0/3798463552 shutdown_connections 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.546+0000 7f692a20d700 1 --2- 172.21.15.132:0/3798463552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 0x7f692407c340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.546+0000 7f692a20d700 1 -- 172.21.15.132:0/3798463552 >> 172.21.15.132:0/3798463552 conn(0x7f6924103a80 msgr2=0x7f6924105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.546+0000 7f692a20d700 1 -- 172.21.15.132:0/3798463552 shutdown_connections 2024-04-23T18:26:40.501 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.546+0000 7f692a20d700 1 -- 172.21.15.132:0/3798463552 wait complete. 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.547+0000 7f692a20d700 1 Processor -- start 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.547+0000 7f692a20d700 1 -- start start 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.547+0000 7f692a20d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 0x7f692419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.547+0000 7f692a20d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f692410e990 con 0x7f692407de90 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.547+0000 7f69237fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 0x7f692419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.547+0000 7f69237fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 0x7f692419e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36658/0 (socket says 172.21.15.132:36658) 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.547+0000 7f69237fe700 1 -- 172.21.15.132:0/320868367 learned_addr learned my addr 172.21.15.132:0/320868367 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.548+0000 7f69237fe700 1 -- 172.21.15.132:0/320868367 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692419eab0 con 0x7f692407de90 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.548+0000 7f69237fe700 1 --2- 172.21.15.132:0/320868367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 0x7f692419e570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6914011ac0 tx=0x7f6914011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.548+0000 7f6920ff9700 1 -- 172.21.15.132:0/320868367 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6914003970 con 0x7f692407de90 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.548+0000 7f692a20d700 1 -- 172.21.15.132:0/320868367 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692419ed40 con 0x7f692407de90 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.548+0000 7f6920ff9700 1 -- 172.21.15.132:0/320868367 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6914003b10 con 0x7f692407de90 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.548+0000 7f6920ff9700 1 -- 172.21.15.132:0/320868367 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6914024610 con 0x7f692407de90 2024-04-23T18:26:40.502 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.548+0000 7f692a20d700 1 -- 172.21.15.132:0/320868367 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69241a1a30 con 0x7f692407de90 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.549+0000 7f6920ff9700 1 -- 172.21.15.132:0/320868367 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6914024890 con 0x7f692407de90 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.549+0000 7f6920ff9700 1 --2- 172.21.15.132:0/320868367 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f690c0395b0 0x7f690c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.549+0000 7f6920ff9700 1 -- 172.21.15.132:0/320868367 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f691404d980 con 0x7f692407de90 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.549+0000 7f6922ffd700 1 --2- 172.21.15.132:0/320868367 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f690c0395b0 0x7f690c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.549+0000 7f692a20d700 1 -- 172.21.15.132:0/320868367 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69240640a0 con 0x7f692407de90 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.549+0000 7f6922ffd700 1 --2- 172.21.15.132:0/320868367 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f690c0395b0 0x7f690c03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6918006fd0 tx=0x7f6918006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.554+0000 7f6920ff9700 1 -- 172.21.15.132:0/320868367 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69140172b0 con 0x7f692407de90 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.793+0000 7f692a20d700 1 -- 172.21.15.132:0/320868367 --> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6924104cc0 con 0x7f690c0395b0 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.798+0000 7f6920ff9700 1 -- 172.21.15.132:0/320868367 <== mgr.14120 v2:172.21.15.132:6800/532999776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6924104cc0 con 0x7f690c0395b0 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.800+0000 7f692a20d700 1 -- 172.21.15.132:0/320868367 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f690c0395b0 msgr2=0x7f690c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.800+0000 7f692a20d700 1 --2- 172.21.15.132:0/320868367 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f690c0395b0 0x7f690c03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6918006fd0 tx=0x7f6918006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.800+0000 7f692a20d700 1 -- 172.21.15.132:0/320868367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 msgr2=0x7f692419e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.800+0000 7f692a20d700 1 --2- 172.21.15.132:0/320868367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 0x7f692419e570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6914011ac0 tx=0x7f6914011ba0 comp rx=0 tx=0).stop 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.800+0000 7f692a20d700 1 -- 172.21.15.132:0/320868367 shutdown_connections 2024-04-23T18:26:40.503 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.800+0000 7f692a20d700 1 --2- 172.21.15.132:0/320868367 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f690c0395b0 0x7f690c03ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:40.504 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.800+0000 7f692a20d700 1 --2- 172.21.15.132:0/320868367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692407de90 0x7f692419e570 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:40.504 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.800+0000 7f692a20d700 1 -- 172.21.15.132:0/320868367 >> 172.21.15.132:0/320868367 conn(0x7f6924103a80 msgr2=0x7f69241045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:40.504 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.800+0000 7f692a20d700 1 -- 172.21.15.132:0/320868367 shutdown_connections 2024-04-23T18:26:40.504 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:39.800+0000 7f692a20d700 1 -- 172.21.15.132:0/320868367 wait complete. 2024-04-23T18:26:40.504 INFO:teuthology.orchestra.run.smithi132.stdout:Deploying unmanaged mgr service... 2024-04-23T18:26:41.080 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:40 smithi132 ceph-mon[29178]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:41.080 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:40 smithi132 ceph-mon[29178]: Saving service mon spec with placement count:5 2024-04-23T18:26:41.080 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:40 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:42.597 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:42 smithi132 ceph-mon[29178]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:42.597 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:42 smithi132 ceph-mon[29178]: Saving service mgr spec with placement count:2 2024-04-23T18:26:42.597 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:42 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:42.597 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:42 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:42.604 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-23T18:26:42.604 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.954+0000 7f863f458700 1 Processor -- start 2024-04-23T18:26:42.604 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.954+0000 7f863f458700 1 -- start start 2024-04-23T18:26:42.604 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.955+0000 7f863f458700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 0x7f8638108830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:42.604 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.955+0000 7f863f458700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8638108e00 con 0x7f8638108410 2024-04-23T18:26:42.604 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.955+0000 7f863d1f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 0x7f8638108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:42.604 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.955+0000 7f863d1f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 0x7f8638108830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36672/0 (socket says 172.21.15.132:36672) 2024-04-23T18:26:42.604 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.955+0000 7f863d1f4700 1 -- 172.21.15.132:0/3315408474 learned_addr learned my addr 172.21.15.132:0/3315408474 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.955+0000 7f863d1f4700 1 -- 172.21.15.132:0/3315408474 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86381095c0 con 0x7f8638108410 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.955+0000 7f863d1f4700 1 --2- 172.21.15.132:0/3315408474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 0x7f8638108830 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f86340098a0 tx=0x7f8634009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915a3e2b31e85a11 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.955+0000 7f862ffff700 1 -- 172.21.15.132:0/3315408474 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8634004160 con 0x7f8638108410 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.955+0000 7f862ffff700 1 -- 172.21.15.132:0/3315408474 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8634004300 con 0x7f8638108410 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.955+0000 7f862ffff700 1 -- 172.21.15.132:0/3315408474 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8634011af0 con 0x7f8638108410 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.956+0000 7f863f458700 1 -- 172.21.15.132:0/3315408474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 msgr2=0x7f8638108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.956+0000 7f863f458700 1 --2- 172.21.15.132:0/3315408474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 0x7f8638108830 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f86340098a0 tx=0x7f8634009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.956+0000 7f863f458700 1 -- 172.21.15.132:0/3315408474 shutdown_connections 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.956+0000 7f863f458700 1 --2- 172.21.15.132:0/3315408474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 0x7f8638108830 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.956+0000 7f863f458700 1 -- 172.21.15.132:0/3315408474 >> 172.21.15.132:0/3315408474 conn(0x7f86381039f0 msgr2=0x7f8638105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.956+0000 7f863f458700 1 -- 172.21.15.132:0/3315408474 shutdown_connections 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.956+0000 7f863f458700 1 -- 172.21.15.132:0/3315408474 wait complete. 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f863f458700 1 Processor -- start 2024-04-23T18:26:42.605 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f863f458700 1 -- start start 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f863f458700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 0x7f863807db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f863f458700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f863810c970 con 0x7f8638108410 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f863d1f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 0x7f863807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f863d1f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 0x7f863807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36680/0 (socket says 172.21.15.132:36680) 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f863d1f4700 1 -- 172.21.15.132:0/1489715421 learned_addr learned my addr 172.21.15.132:0/1489715421 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f863d1f4700 1 -- 172.21.15.132:0/1489715421 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863807e0d0 con 0x7f8638108410 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f863d1f4700 1 --2- 172.21.15.132:0/1489715421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 0x7f863807db90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8634000c00 tx=0x7f8634011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f862e7fc700 1 -- 172.21.15.132:0/1489715421 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8634003b20 con 0x7f8638108410 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f862e7fc700 1 -- 172.21.15.132:0/1489715421 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f863401a8c0 con 0x7f8638108410 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f862e7fc700 1 -- 172.21.15.132:0/1489715421 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8634024940 con 0x7f8638108410 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f863f458700 1 -- 172.21.15.132:0/1489715421 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863807c2c0 con 0x7f8638108410 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.957+0000 7f863f458700 1 -- 172.21.15.132:0/1489715421 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863807c800 con 0x7f8638108410 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.958+0000 7f862e7fc700 1 -- 172.21.15.132:0/1489715421 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8634022020 con 0x7f8638108410 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.958+0000 7f863f458700 1 -- 172.21.15.132:0/1489715421 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86380640a0 con 0x7f8638108410 2024-04-23T18:26:42.606 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.959+0000 7f862e7fc700 1 --2- 172.21.15.132:0/1489715421 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f8624039600 0x7f862403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.959+0000 7f862e7fc700 1 -- 172.21.15.132:0/1489715421 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f863404d910 con 0x7f8638108410 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.959+0000 7f863c9f3700 1 --2- 172.21.15.132:0/1489715421 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f8624039600 0x7f862403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.959+0000 7f863c9f3700 1 --2- 172.21.15.132:0/1489715421 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f8624039600 0x7f862403bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8628006fd0 tx=0x7f8628006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:40.964+0000 7f862e7fc700 1 -- 172.21.15.132:0/1489715421 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f863401aae0 con 0x7f8638108410 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:41.183+0000 7f863f458700 1 -- 172.21.15.132:0/1489715421 --> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8638104c40 con 0x7f8624039600 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:41.191+0000 7f862e7fc700 1 -- 172.21.15.132:0/1489715421 <== mgr.14120 v2:172.21.15.132:6800/532999776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8638104c40 con 0x7f8624039600 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:41.193+0000 7f863f458700 1 -- 172.21.15.132:0/1489715421 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f8624039600 msgr2=0x7f862403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:41.193+0000 7f863f458700 1 --2- 172.21.15.132:0/1489715421 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f8624039600 0x7f862403bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8628006fd0 tx=0x7f8628006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:41.193+0000 7f863f458700 1 -- 172.21.15.132:0/1489715421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 msgr2=0x7f863807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:41.193+0000 7f863f458700 1 --2- 172.21.15.132:0/1489715421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 0x7f863807db90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8634000c00 tx=0x7f8634011b70 comp rx=0 tx=0).stop 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:41.193+0000 7f863f458700 1 -- 172.21.15.132:0/1489715421 shutdown_connections 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:41.193+0000 7f863f458700 1 --2- 172.21.15.132:0/1489715421 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f8624039600 0x7f862403bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:41.193+0000 7f863f458700 1 --2- 172.21.15.132:0/1489715421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8638108410 0x7f863807db90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:41.193+0000 7f863f458700 1 -- 172.21.15.132:0/1489715421 >> 172.21.15.132:0/1489715421 conn(0x7f86381039f0 msgr2=0x7f8638104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:41.193+0000 7f863f458700 1 -- 172.21.15.132:0/1489715421 shutdown_connections 2024-04-23T18:26:42.607 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:41.193+0000 7f863f458700 1 -- 172.21.15.132:0/1489715421 wait complete. 2024-04-23T18:26:43.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:43 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:43.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:43 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3538699592' entity='client.admin' 2024-04-23T18:26:44.557 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.070+0000 7fa056cc9700 1 Processor -- start 2024-04-23T18:26:44.557 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.070+0000 7fa056cc9700 1 -- start start 2024-04-23T18:26:44.557 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.070+0000 7fa056cc9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 0x7fa05010b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:44.557 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.070+0000 7fa056cc9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa05007be90 con 0x7fa050108ea0 2024-04-23T18:26:44.557 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.070+0000 7fa054a65700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 0x7fa05010b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:44.557 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.071+0000 7fa054a65700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 0x7fa05010b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36692/0 (socket says 172.21.15.132:36692) 2024-04-23T18:26:44.557 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.071+0000 7fa054a65700 1 -- 172.21.15.132:0/3820995697 learned_addr learned my addr 172.21.15.132:0/3820995697 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:44.557 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.071+0000 7fa054a65700 1 -- 172.21.15.132:0/3820995697 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05010b7d0 con 0x7fa050108ea0 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.071+0000 7fa054a65700 1 --2- 172.21.15.132:0/3820995697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 0x7fa05010b290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0440098a0 tx=0x7fa044009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7085deaa87a55873 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.071+0000 7fa04f7fe700 1 -- 172.21.15.132:0/3820995697 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa044004160 con 0x7fa050108ea0 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.071+0000 7fa04f7fe700 1 -- 172.21.15.132:0/3820995697 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa044004300 con 0x7fa050108ea0 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.071+0000 7fa04f7fe700 1 -- 172.21.15.132:0/3820995697 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa044011a20 con 0x7fa050108ea0 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.072+0000 7fa056cc9700 1 -- 172.21.15.132:0/3820995697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 msgr2=0x7fa05010b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.072+0000 7fa056cc9700 1 --2- 172.21.15.132:0/3820995697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 0x7fa05010b290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0440098a0 tx=0x7fa044009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.072+0000 7fa056cc9700 1 -- 172.21.15.132:0/3820995697 shutdown_connections 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.072+0000 7fa056cc9700 1 --2- 172.21.15.132:0/3820995697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 0x7fa05010b290 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.072+0000 7fa056cc9700 1 -- 172.21.15.132:0/3820995697 >> 172.21.15.132:0/3820995697 conn(0x7fa050102890 msgr2=0x7fa050104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.072+0000 7fa056cc9700 1 -- 172.21.15.132:0/3820995697 shutdown_connections 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.072+0000 7fa056cc9700 1 -- 172.21.15.132:0/3820995697 wait complete. 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.073+0000 7fa056cc9700 1 Processor -- start 2024-04-23T18:26:44.558 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.073+0000 7fa056cc9700 1 -- start start 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.073+0000 7fa056cc9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 0x7fa05019c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.073+0000 7fa056cc9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010c7d0 con 0x7fa050108ea0 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.073+0000 7fa054a65700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 0x7fa05019c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.073+0000 7fa054a65700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 0x7fa05019c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36694/0 (socket says 172.21.15.132:36694) 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.073+0000 7fa054a65700 1 -- 172.21.15.132:0/3538699592 learned_addr learned my addr 172.21.15.132:0/3538699592 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.073+0000 7fa054a65700 1 -- 172.21.15.132:0/3538699592 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05019c860 con 0x7fa050108ea0 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.074+0000 7fa054a65700 1 --2- 172.21.15.132:0/3538699592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 0x7fa05019c320 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa044004660 tx=0x7fa044011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.074+0000 7fa04dffb700 1 -- 172.21.15.132:0/3538699592 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa044003aa0 con 0x7fa050108ea0 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.074+0000 7fa04dffb700 1 -- 172.21.15.132:0/3538699592 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa04401b8f0 con 0x7fa050108ea0 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.074+0000 7fa056cc9700 1 -- 172.21.15.132:0/3538699592 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05019caf0 con 0x7fa050108ea0 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.074+0000 7fa04dffb700 1 -- 172.21.15.132:0/3538699592 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa044023900 con 0x7fa050108ea0 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.074+0000 7fa056cc9700 1 -- 172.21.15.132:0/3538699592 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05019f7e0 con 0x7fa050108ea0 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.075+0000 7fa04dffb700 1 -- 172.21.15.132:0/3538699592 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa044023b70 con 0x7fa050108ea0 2024-04-23T18:26:44.559 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.075+0000 7fa056cc9700 1 -- 172.21.15.132:0/3538699592 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0500640a0 con 0x7fa050108ea0 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.075+0000 7fa04dffb700 1 --2- 172.21.15.132:0/3538699592 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa040039600 0x7fa04003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.075+0000 7fa04dffb700 1 -- 172.21.15.132:0/3538699592 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa04404de40 con 0x7fa050108ea0 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.075+0000 7fa04ffff700 1 --2- 172.21.15.132:0/3538699592 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa040039600 0x7fa04003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.076+0000 7fa04ffff700 1 --2- 172.21.15.132:0/3538699592 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa040039600 0x7fa04003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa03c006fd0 tx=0x7fa03c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.080+0000 7fa04dffb700 1 -- 172.21.15.132:0/3538699592 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa04401f030 con 0x7fa050108ea0 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.311+0000 7fa056cc9700 1 -- 172.21.15.132:0/3538699592 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fa05019fdd0 con 0x7fa050108ea0 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.313+0000 7fa04dffb700 1 -- 172.21.15.132:0/3538699592 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fa04401bde0 con 0x7fa050108ea0 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.316+0000 7fa056cc9700 1 -- 172.21.15.132:0/3538699592 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa040039600 msgr2=0x7fa04003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.316+0000 7fa056cc9700 1 --2- 172.21.15.132:0/3538699592 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa040039600 0x7fa04003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa03c006fd0 tx=0x7fa03c006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.316+0000 7fa056cc9700 1 -- 172.21.15.132:0/3538699592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 msgr2=0x7fa05019c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.316+0000 7fa056cc9700 1 --2- 172.21.15.132:0/3538699592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 0x7fa05019c320 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa044004660 tx=0x7fa044011aa0 comp rx=0 tx=0).stop 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.316+0000 7fa056cc9700 1 -- 172.21.15.132:0/3538699592 shutdown_connections 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.316+0000 7fa056cc9700 1 --2- 172.21.15.132:0/3538699592 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa040039600 0x7fa04003bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.316+0000 7fa056cc9700 1 --2- 172.21.15.132:0/3538699592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050108ea0 0x7fa05019c320 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:44.561 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.316+0000 7fa056cc9700 1 -- 172.21.15.132:0/3538699592 >> 172.21.15.132:0/3538699592 conn(0x7fa050102890 msgr2=0x7fa050103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:44.561 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.316+0000 7fa056cc9700 1 -- 172.21.15.132:0/3538699592 shutdown_connections 2024-04-23T18:26:44.561 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:43.316+0000 7fa056cc9700 1 -- 172.21.15.132:0/3538699592 wait complete. 2024-04-23T18:26:45.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:44 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:45.763 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.935+0000 7f70b213a700 1 Processor -- start 2024-04-23T18:26:45.763 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.936+0000 7f70b213a700 1 -- start start 2024-04-23T18:26:45.763 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.936+0000 7f70b213a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac07de90 0x7f70ac07c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.763 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.936+0000 7f70b213a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac07c910 con 0x7f70ac07de90 2024-04-23T18:26:45.763 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.936+0000 7f70ab7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac07de90 0x7f70ac07c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.763 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.936+0000 7f70ab7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac07de90 0x7f70ac07c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36704/0 (socket says 172.21.15.132:36704) 2024-04-23T18:26:45.763 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.936+0000 7f70ab7fe700 1 -- 172.21.15.132:0/4043166610 learned_addr learned my addr 172.21.15.132:0/4043166610 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:45.763 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.936+0000 7f70ab7fe700 1 -- 172.21.15.132:0/4043166610 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac07ca90 con 0x7f70ac07de90 2024-04-23T18:26:45.763 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.936+0000 7f70ab7fe700 1 --2- 172.21.15.132:0/4043166610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac07de90 0x7f70ac07c340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f709c0098a0 tx=0x7f709c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff5962f28d87d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.936+0000 7f70aaffd700 1 -- 172.21.15.132:0/4043166610 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f709c004160 con 0x7f70ac07de90 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.936+0000 7f70aaffd700 1 -- 172.21.15.132:0/4043166610 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f709c004300 con 0x7f70ac07de90 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.937+0000 7f70aaffd700 1 -- 172.21.15.132:0/4043166610 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f709c011a20 con 0x7f70ac07de90 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.937+0000 7f70b213a700 1 -- 172.21.15.132:0/4043166610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac07de90 msgr2=0x7f70ac07c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.937+0000 7f70b213a700 1 --2- 172.21.15.132:0/4043166610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac07de90 0x7f70ac07c340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f709c0098a0 tx=0x7f709c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.937+0000 7f70b213a700 1 -- 172.21.15.132:0/4043166610 shutdown_connections 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.937+0000 7f70b213a700 1 --2- 172.21.15.132:0/4043166610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac07de90 0x7f70ac07c340 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.937+0000 7f70b213a700 1 -- 172.21.15.132:0/4043166610 >> 172.21.15.132:0/4043166610 conn(0x7f70ac103ac0 msgr2=0x7f70ac105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.937+0000 7f70b213a700 1 -- 172.21.15.132:0/4043166610 shutdown_connections 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.937+0000 7f70b213a700 1 -- 172.21.15.132:0/4043166610 wait complete. 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.938+0000 7f70b213a700 1 Processor -- start 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.938+0000 7f70b213a700 1 -- start start 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.938+0000 7f70b213a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac19e740 0x7f70ac19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.764 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.938+0000 7f70b213a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac10ea20 con 0x7f70ac19e740 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.938+0000 7f70ab7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac19e740 0x7f70ac19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.938+0000 7f70ab7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac19e740 0x7f70ac19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36718/0 (socket says 172.21.15.132:36718) 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.938+0000 7f70ab7fe700 1 -- 172.21.15.132:0/2891341479 learned_addr learned my addr 172.21.15.132:0/2891341479 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.938+0000 7f70ab7fe700 1 -- 172.21.15.132:0/2891341479 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac19f0a0 con 0x7f70ac19e740 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.938+0000 7f70ab7fe700 1 --2- 172.21.15.132:0/2891341479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac19e740 0x7f70ac19eb60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f709c000c00 tx=0x7f709c012fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.939+0000 7f70a97fa700 1 -- 172.21.15.132:0/2891341479 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f709c011a20 con 0x7f70ac19e740 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.939+0000 7f70b213a700 1 -- 172.21.15.132:0/2891341479 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ac19f330 con 0x7f70ac19e740 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.939+0000 7f70a97fa700 1 -- 172.21.15.132:0/2891341479 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f709c003b50 con 0x7f70ac19e740 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.939+0000 7f70a97fa700 1 -- 172.21.15.132:0/2891341479 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f709c025db0 con 0x7f70ac19e740 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.939+0000 7f70b213a700 1 -- 172.21.15.132:0/2891341479 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ac1a1e10 con 0x7f70ac19e740 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.939+0000 7f70a97fa700 1 -- 172.21.15.132:0/2891341479 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f709c01b450 con 0x7f70ac19e740 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.940+0000 7f70a97fa700 1 --2- 172.21.15.132:0/2891341479 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7098039560 0x7f709803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.940+0000 7f70b213a700 1 -- 172.21.15.132:0/2891341479 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70ac0640a0 con 0x7f70ac19e740 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.940+0000 7f70a97fa700 1 -- 172.21.15.132:0/2891341479 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f709c04dcd0 con 0x7f70ac19e740 2024-04-23T18:26:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.940+0000 7f70a2ffd700 1 --2- 172.21.15.132:0/2891341479 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7098039560 0x7f709803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.940+0000 7f70a2ffd700 1 --2- 172.21.15.132:0/2891341479 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7098039560 0x7f709803ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7094006fd0 tx=0x7f7094006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:44.945+0000 7f70a97fa700 1 -- 172.21.15.132:0/2891341479 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f709c02c2f0 con 0x7f70ac19e740 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:45.147+0000 7f70b213a700 1 -- 172.21.15.132:0/2891341479 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f70ac1a2620 con 0x7f70ac19e740 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:45.150+0000 7f70a97fa700 1 -- 172.21.15.132:0/2891341479 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f709c028e00 con 0x7f70ac19e740 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:45.153+0000 7f70b213a700 1 -- 172.21.15.132:0/2891341479 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7098039560 msgr2=0x7f709803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:45.153+0000 7f70b213a700 1 --2- 172.21.15.132:0/2891341479 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7098039560 0x7f709803ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7094006fd0 tx=0x7f7094006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:45.153+0000 7f70b213a700 1 -- 172.21.15.132:0/2891341479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac19e740 msgr2=0x7f70ac19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:45.153+0000 7f70b213a700 1 --2- 172.21.15.132:0/2891341479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac19e740 0x7f70ac19eb60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f709c000c00 tx=0x7f709c012fc0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:45.153+0000 7f70b213a700 1 -- 172.21.15.132:0/2891341479 shutdown_connections 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:45.153+0000 7f70b213a700 1 --2- 172.21.15.132:0/2891341479 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7098039560 0x7f709803ba20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:45.153+0000 7f70b213a700 1 --2- 172.21.15.132:0/2891341479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70ac19e740 0x7f70ac19eb60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:45.153+0000 7f70b213a700 1 -- 172.21.15.132:0/2891341479 >> 172.21.15.132:0/2891341479 conn(0x7f70ac103ac0 msgr2=0x7f70ac104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:45.153+0000 7f70b213a700 1 -- 172.21.15.132:0/2891341479 shutdown_connections 2024-04-23T18:26:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:45.153+0000 7f70b213a700 1 -- 172.21.15.132:0/2891341479 wait complete. 2024-04-23T18:26:45.767 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling the dashboard module... 2024-04-23T18:26:46.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:46 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2891341479' entity='client.admin' 2024-04-23T18:26:46.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:46 smithi132 ceph-mon[29178]: from='mgr.14120 172.21.15.132:0/3007543588' entity='mgr.x' 2024-04-23T18:26:47.507 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:47 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2640639440' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-23T18:26:47.508 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: ignoring --setuser ceph since I am not root 2024-04-23T18:26:47.508 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: ignoring --setgroup ceph since I am not root 2024-04-23T18:26:47.508 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:47.331+0000 7f6c585cd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:26:47.508 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:47.514+0000 7f6c585cd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:47.862 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:47.619+0000 7f6c585cd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:26:47.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.220+0000 7fcaf9c2c700 1 Processor -- start 2024-04-23T18:26:47.900 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.220+0000 7fcaf9c2c700 1 -- start start 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.220+0000 7fcaf9c2c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 0x7fcaf410b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.220+0000 7fcaf9c2c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf407beb0 con 0x7fcaf4108ef0 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.221+0000 7fcaf37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 0x7fcaf410b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.221+0000 7fcaf37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 0x7fcaf410b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36722/0 (socket says 172.21.15.132:36722) 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.221+0000 7fcaf37fe700 1 -- 172.21.15.132:0/3188796271 learned_addr learned my addr 172.21.15.132:0/3188796271 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.221+0000 7fcaf37fe700 1 -- 172.21.15.132:0/3188796271 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf410b820 con 0x7fcaf4108ef0 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.221+0000 7fcaf37fe700 1 --2- 172.21.15.132:0/3188796271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 0x7fcaf410b2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcae80098a0 tx=0x7fcae8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924fe6cdae364186 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.221+0000 7fcaf27fc700 1 -- 172.21.15.132:0/3188796271 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcae8004160 con 0x7fcaf4108ef0 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.221+0000 7fcaf27fc700 1 -- 172.21.15.132:0/3188796271 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae8004300 con 0x7fcaf4108ef0 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.221+0000 7fcaf27fc700 1 -- 172.21.15.132:0/3188796271 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcae8011a20 con 0x7fcaf4108ef0 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.222+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/3188796271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 msgr2=0x7fcaf410b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.222+0000 7fcaf9c2c700 1 --2- 172.21.15.132:0/3188796271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 0x7fcaf410b2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcae80098a0 tx=0x7fcae8009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:47.901 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.222+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/3188796271 shutdown_connections 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.222+0000 7fcaf9c2c700 1 --2- 172.21.15.132:0/3188796271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 0x7fcaf410b2e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.222+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/3188796271 >> 172.21.15.132:0/3188796271 conn(0x7fcaf4102920 msgr2=0x7fcaf4104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.222+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/3188796271 shutdown_connections 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.222+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/3188796271 wait complete. 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.223+0000 7fcaf9c2c700 1 Processor -- start 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.223+0000 7fcaf9c2c700 1 -- start start 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.223+0000 7fcaf9c2c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 0x7fcaf419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.223+0000 7fcaf9c2c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf410c7c0 con 0x7fcaf4108ef0 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.223+0000 7fcaf37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 0x7fcaf419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.223+0000 7fcaf37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 0x7fcaf419c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36738/0 (socket says 172.21.15.132:36738) 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.223+0000 7fcaf37fe700 1 -- 172.21.15.132:0/2640639440 learned_addr learned my addr 172.21.15.132:0/2640639440 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.223+0000 7fcaf37fe700 1 -- 172.21.15.132:0/2640639440 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf419c850 con 0x7fcaf4108ef0 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.224+0000 7fcaf37fe700 1 --2- 172.21.15.132:0/2640639440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 0x7fcaf419c310 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcae80116e0 tx=0x7fcae8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:47.902 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.224+0000 7fcaf0ff9700 1 -- 172.21.15.132:0/2640639440 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcae8003b20 con 0x7fcaf4108ef0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.224+0000 7fcaf0ff9700 1 -- 172.21.15.132:0/2640639440 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae801b8f0 con 0x7fcaf4108ef0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.224+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/2640639440 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf419cb40 con 0x7fcaf4108ef0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.224+0000 7fcaf0ff9700 1 -- 172.21.15.132:0/2640639440 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcae80238c0 con 0x7fcaf4108ef0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.224+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/2640639440 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf419f7d0 con 0x7fcaf4108ef0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.225+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/2640639440 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf40640a0 con 0x7fcaf4108ef0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.225+0000 7fcaf0ff9700 1 -- 172.21.15.132:0/2640639440 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fcae8023b70 con 0x7fcaf4108ef0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.225+0000 7fcaf0ff9700 1 --2- 172.21.15.132:0/2640639440 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fcadc039560 0x7fcadc03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.225+0000 7fcaf0ff9700 1 -- 172.21.15.132:0/2640639440 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcae804e010 con 0x7fcaf4108ef0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.225+0000 7fcaf2ffd700 1 --2- 172.21.15.132:0/2640639440 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fcadc039560 0x7fcadc03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.226+0000 7fcaf2ffd700 1 --2- 172.21.15.132:0/2640639440 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fcadc039560 0x7fcadc03ba20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcae4006fd0 tx=0x7fcae4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.230+0000 7fcaf0ff9700 1 -- 172.21.15.132:0/2640639440 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcae801f030 con 0x7fcaf4108ef0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:46.518+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/2640639440 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fcaf419fa00 con 0x7fcaf4108ef0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:47.153+0000 7fcaf0ff9700 1 -- 172.21.15.132:0/2640639440 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fcae8018e30 con 0x7fcaf4108ef0 2024-04-23T18:26:47.903 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:47.154+0000 7fcaf0ff9700 1 -- 172.21.15.132:0/2640639440 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fcae801baa0 con 0x7fcaf4108ef0 2024-04-23T18:26:47.904 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:47.156+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/2640639440 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fcadc039560 msgr2=0x7fcadc03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:47.904 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:47.156+0000 7fcaf9c2c700 1 --2- 172.21.15.132:0/2640639440 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fcadc039560 0x7fcadc03ba20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcae4006fd0 tx=0x7fcae4006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:47.904 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:47.156+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/2640639440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 msgr2=0x7fcaf419c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:47.904 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:47.156+0000 7fcaf9c2c700 1 --2- 172.21.15.132:0/2640639440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 0x7fcaf419c310 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcae80116e0 tx=0x7fcae8011aa0 comp rx=0 tx=0).stop 2024-04-23T18:26:47.904 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:47.156+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/2640639440 shutdown_connections 2024-04-23T18:26:47.904 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:47.156+0000 7fcaf9c2c700 1 --2- 172.21.15.132:0/2640639440 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fcadc039560 0x7fcadc03ba20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:47.904 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:47.156+0000 7fcaf9c2c700 1 --2- 172.21.15.132:0/2640639440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4108ef0 0x7fcaf419c310 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:47.904 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:47.156+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/2640639440 >> 172.21.15.132:0/2640639440 conn(0x7fcaf4102920 msgr2=0x7fcaf41033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:47.904 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:47.157+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/2640639440 shutdown_connections 2024-04-23T18:26:47.904 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:47.157+0000 7fcaf9c2c700 1 -- 172.21.15.132:0/2640639440 wait complete. 2024-04-23T18:26:48.146 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:47.930+0000 7f6c585cd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:26:48.147 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:48 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:48.017+0000 7f6c585cd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:26:48.553 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:48 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2640639440' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-23T18:26:48.553 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:48 smithi132 ceph-mon[29178]: mgrmap e9: x(active, since 19s) 2024-04-23T18:26:49.266 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:49 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3847473713' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T18:26:49.266 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:49 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:49.229+0000 7f6c585cd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:26:49.275 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.353+0000 7f748093c700 1 Processor -- start 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.353+0000 7f748093c700 1 -- start start 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.353+0000 7f748093c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 0x7f747c10b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.353+0000 7f748093c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f747c07beb0 con 0x7f747c108e80 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.354+0000 7f747a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 0x7f747c10b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.354+0000 7f747a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 0x7f747c10b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36770/0 (socket says 172.21.15.132:36770) 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.354+0000 7f747a59c700 1 -- 172.21.15.132:0/337361036 learned_addr learned my addr 172.21.15.132:0/337361036 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.354+0000 7f747a59c700 1 -- 172.21.15.132:0/337361036 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747c10b7b0 con 0x7f747c108e80 2024-04-23T18:26:49.276 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.354+0000 7f747a59c700 1 --2- 172.21.15.132:0/337361036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 0x7f747c10b270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f74700098a0 tx=0x7f7470009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2478d68660b53d71 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.355+0000 7f747959a700 1 -- 172.21.15.132:0/337361036 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7470004160 con 0x7f747c108e80 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.355+0000 7f747959a700 1 -- 172.21.15.132:0/337361036 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7470004300 con 0x7f747c108e80 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.355+0000 7f747959a700 1 -- 172.21.15.132:0/337361036 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7470011a20 con 0x7f747c108e80 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.355+0000 7f748093c700 1 -- 172.21.15.132:0/337361036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 msgr2=0x7f747c10b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.355+0000 7f748093c700 1 --2- 172.21.15.132:0/337361036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 0x7f747c10b270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f74700098a0 tx=0x7f7470009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.355+0000 7f748093c700 1 -- 172.21.15.132:0/337361036 shutdown_connections 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.355+0000 7f748093c700 1 --2- 172.21.15.132:0/337361036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 0x7f747c10b270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.355+0000 7f748093c700 1 -- 172.21.15.132:0/337361036 >> 172.21.15.132:0/337361036 conn(0x7f747c102890 msgr2=0x7f747c104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.355+0000 7f748093c700 1 -- 172.21.15.132:0/337361036 shutdown_connections 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.355+0000 7f748093c700 1 -- 172.21.15.132:0/337361036 wait complete. 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.356+0000 7f748093c700 1 Processor -- start 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.356+0000 7f748093c700 1 -- start start 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.356+0000 7f748093c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 0x7f747c19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.356+0000 7f748093c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f747c10c750 con 0x7f747c108e80 2024-04-23T18:26:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.356+0000 7f747a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 0x7f747c19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.356+0000 7f747a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 0x7f747c19c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36782/0 (socket says 172.21.15.132:36782) 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.356+0000 7f747a59c700 1 -- 172.21.15.132:0/3847473713 learned_addr learned my addr 172.21.15.132:0/3847473713 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.357+0000 7f747a59c700 1 -- 172.21.15.132:0/3847473713 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747c19c810 con 0x7f747c108e80 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.357+0000 7f747a59c700 1 --2- 172.21.15.132:0/3847473713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 0x7f747c19c2d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7470004660 tx=0x7f7470011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.357+0000 7f746b7fe700 1 -- 172.21.15.132:0/3847473713 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7470003aa0 con 0x7f747c108e80 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.357+0000 7f746b7fe700 1 -- 172.21.15.132:0/3847473713 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f747001b8f0 con 0x7f747c108e80 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.357+0000 7f746b7fe700 1 -- 172.21.15.132:0/3847473713 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7470023900 con 0x7f747c108e80 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.357+0000 7f748093c700 1 -- 172.21.15.132:0/3847473713 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747c19cb00 con 0x7f747c108e80 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.357+0000 7f748093c700 1 -- 172.21.15.132:0/3847473713 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f747c19f790 con 0x7f747c108e80 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.358+0000 7f748093c700 1 -- 172.21.15.132:0/3847473713 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f747c0640a0 con 0x7f747c108e80 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.358+0000 7f746b7fe700 1 -- 172.21.15.132:0/3847473713 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f7470023b70 con 0x7f747c108e80 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.359+0000 7f746b7fe700 1 --2- 172.21.15.132:0/3847473713 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7464039660 0x7f746403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.278 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.359+0000 7f746b7fe700 1 -- 172.21.15.132:0/3847473713 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f747004dfc0 con 0x7f747c108e80 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.359+0000 7f7479d9b700 1 -- 172.21.15.132:0/3847473713 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7464039660 msgr2=0x7f746403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/532999776 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.359+0000 7f7479d9b700 1 --2- 172.21.15.132:0/3847473713 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7464039660 0x7f746403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.363+0000 7f746b7fe700 1 -- 172.21.15.132:0/3847473713 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f747001f030 con 0x7f747c108e80 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.560+0000 7f7479d9b700 1 -- 172.21.15.132:0/3847473713 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7464039660 msgr2=0x7f746403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/532999776 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.560+0000 7f7479d9b700 1 --2- 172.21.15.132:0/3847473713 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7464039660 0x7f746403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.657+0000 7f748093c700 1 -- 172.21.15.132:0/3847473713 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f747c19fa10 con 0x7f747c108e80 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.658+0000 7f746b7fe700 1 -- 172.21.15.132:0/3847473713 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f747001bde0 con 0x7f747c108e80 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.660+0000 7f748093c700 1 -- 172.21.15.132:0/3847473713 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7464039660 msgr2=0x7f746403bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.660+0000 7f748093c700 1 --2- 172.21.15.132:0/3847473713 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7464039660 0x7f746403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.660+0000 7f748093c700 1 -- 172.21.15.132:0/3847473713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 msgr2=0x7f747c19c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.660+0000 7f748093c700 1 --2- 172.21.15.132:0/3847473713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 0x7f747c19c2d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7470004660 tx=0x7f7470011aa0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.660+0000 7f748093c700 1 -- 172.21.15.132:0/3847473713 shutdown_connections 2024-04-23T18:26:49.279 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.660+0000 7f748093c700 1 --2- 172.21.15.132:0/3847473713 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7f7464039660 0x7f746403bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.280 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.660+0000 7f748093c700 1 --2- 172.21.15.132:0/3847473713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f747c108e80 0x7f747c19c2d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.280 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.660+0000 7f748093c700 1 -- 172.21.15.132:0/3847473713 >> 172.21.15.132:0/3847473713 conn(0x7f747c102890 msgr2=0x7f747c1033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:49.280 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.660+0000 7f748093c700 1 -- 172.21.15.132:0/3847473713 shutdown_connections 2024-04-23T18:26:49.280 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:48.660+0000 7f748093c700 1 -- 172.21.15.132:0/3847473713 wait complete. 2024-04-23T18:26:49.280 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for the mgr to restart... 2024-04-23T18:26:49.280 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for mgr epoch 9... 2024-04-23T18:26:49.520 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:49 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:49.411+0000 7f6c585cd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:49.862 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:49 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:49.589+0000 7f6c585cd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:26:50.175 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:49 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:49.885+0000 7f6c585cd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:26:50.175 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:49 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:49.979+0000 7f6c585cd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:26:50.176 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:50 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:50.182+0000 7f6c585cd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:26:50.612 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:50 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:50.376+0000 7f6c585cd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:51.362 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:50 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:50.965+0000 7f6c585cd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:26:51.668 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:51 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:51.381+0000 7f6c585cd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:26:51.668 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:51 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:51.476+0000 7f6c585cd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:26:51.668 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:51 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:51.675+0000 7f6c585cd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:26:52.515 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:52 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:52.191+0000 7f6c585cd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:26:52.515 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:52 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:52.282+0000 7f6c585cd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:26:52.784 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:52 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:52.522+0000 7f6c585cd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:26:52.784 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:52 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:52.660+0000 7f6c585cd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:26:52.785 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:52 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:52.760+0000 7f6c585cd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:26:53.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:52 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:52.884+0000 7f6c585cd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:53.612 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:53 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:53.200+0000 7f6c585cd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:26:54.033 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:53 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:53.823+0000 7f6c585cd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:26:54.033 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:26:54.040+0000 7f6c585cd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:26:54.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-mon[29178]: Active manager daemon x restarted 2024-04-23T18:26:54.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-mon[29178]: Activating manager daemon x 2024-04-23T18:26:54.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-mon[29178]: osdmap e3: 0 total, 0 up, 0 in 2024-04-23T18:26:54.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-mon[29178]: mgrmap e10: x(active, starting, since 0.00341295s) 2024-04-23T18:26:54.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:26:54.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:26:54.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:26:54.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:26:54.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:26:54.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-mon[29178]: Manager daemon x is now available 2024-04-23T18:26:54.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:54.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:26:55.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:26:55.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:55 smithi132 ceph-mon[29178]: mgrmap e11: x(active, since 1.00611s) 2024-04-23T18:26:55.984 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.735+0000 7fa2f63cb700 1 Processor -- start 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.735+0000 7fa2f63cb700 1 -- start start 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.735+0000 7fa2f63cb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 0x7fa2f01088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.735+0000 7fa2f63cb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f0108e70 con 0x7fa2f0108480 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.735+0000 7fa2effff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 0x7fa2f01088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.735+0000 7fa2effff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 0x7fa2f01088a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55308/0 (socket says 172.21.15.132:55308) 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.735+0000 7fa2effff700 1 -- 172.21.15.132:0/3949907766 learned_addr learned my addr 172.21.15.132:0/3949907766 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.736+0000 7fa2effff700 1 -- 172.21.15.132:0/3949907766 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f0109630 con 0x7fa2f0108480 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.736+0000 7fa2effff700 1 --2- 172.21.15.132:0/3949907766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 0x7fa2f01088a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00098a0 tx=0x7fa2e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea67af8750ce9499 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:55.985 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.736+0000 7fa2eeffd700 1 -- 172.21.15.132:0/3949907766 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2e0004160 con 0x7fa2f0108480 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.736+0000 7fa2eeffd700 1 -- 172.21.15.132:0/3949907766 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e0004300 con 0x7fa2f0108480 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.736+0000 7fa2eeffd700 1 -- 172.21.15.132:0/3949907766 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2e0011a20 con 0x7fa2f0108480 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.736+0000 7fa2f63cb700 1 -- 172.21.15.132:0/3949907766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 msgr2=0x7fa2f01088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.736+0000 7fa2f63cb700 1 --2- 172.21.15.132:0/3949907766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 0x7fa2f01088a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00098a0 tx=0x7fa2e0009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.737+0000 7fa2f63cb700 1 -- 172.21.15.132:0/3949907766 shutdown_connections 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.737+0000 7fa2f63cb700 1 --2- 172.21.15.132:0/3949907766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 0x7fa2f01088a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.737+0000 7fa2f63cb700 1 -- 172.21.15.132:0/3949907766 >> 172.21.15.132:0/3949907766 conn(0x7fa2f0103ac0 msgr2=0x7fa2f0105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.737+0000 7fa2f63cb700 1 -- 172.21.15.132:0/3949907766 shutdown_connections 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.737+0000 7fa2f63cb700 1 -- 172.21.15.132:0/3949907766 wait complete. 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.737+0000 7fa2f63cb700 1 Processor -- start 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.737+0000 7fa2f63cb700 1 -- start start 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.738+0000 7fa2f63cb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 0x7fa2f019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:55.986 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.738+0000 7fa2f63cb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f010c9e0 con 0x7fa2f0108480 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.738+0000 7fa2effff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 0x7fa2f019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.738+0000 7fa2effff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 0x7fa2f019c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55322/0 (socket says 172.21.15.132:55322) 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.738+0000 7fa2effff700 1 -- 172.21.15.132:0/4036961442 learned_addr learned my addr 172.21.15.132:0/4036961442 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.738+0000 7fa2effff700 1 -- 172.21.15.132:0/4036961442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f019c8a0 con 0x7fa2f0108480 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.738+0000 7fa2effff700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 0x7fa2f019c360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0000c00 tx=0x7fa2e0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.738+0000 7fa2ed7fa700 1 -- 172.21.15.132:0/4036961442 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2e0003aa0 con 0x7fa2f0108480 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.738+0000 7fa2f63cb700 1 -- 172.21.15.132:0/4036961442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f019cb30 con 0x7fa2f0108480 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.738+0000 7fa2ed7fa700 1 -- 172.21.15.132:0/4036961442 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e001b8f0 con 0x7fa2f0108480 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.738+0000 7fa2ed7fa700 1 -- 172.21.15.132:0/4036961442 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2e0023890 con 0x7fa2f0108480 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.738+0000 7fa2f63cb700 1 -- 172.21.15.132:0/4036961442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f019f820 con 0x7fa2f0108480 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.739+0000 7fa2ed7fa700 1 -- 172.21.15.132:0/4036961442 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fa2e002c3f0 con 0x7fa2f0108480 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.739+0000 7fa2ed7fa700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 0x7fa2d803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.739+0000 7fa2ed7fa700 1 -- 172.21.15.132:0/4036961442 --> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa2d803c230 con 0x7fa2d8039660 2024-04-23T18:26:55.987 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.740+0000 7fa2ed7fa700 1 -- 172.21.15.132:0/4036961442 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa2e004e060 con 0x7fa2f0108480 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.740+0000 7fa2ef7fe700 1 -- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 msgr2=0x7fa2d803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/532999776 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.740+0000 7fa2ef7fe700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 0x7fa2d803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.941+0000 7fa2ef7fe700 1 -- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 msgr2=0x7fa2d803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/532999776 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:49.941+0000 7fa2ef7fe700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 0x7fa2d803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:50.342+0000 7fa2ef7fe700 1 -- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 msgr2=0x7fa2d803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/532999776 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:50.342+0000 7fa2ef7fe700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 0x7fa2d803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:51.143+0000 7fa2ef7fe700 1 -- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 msgr2=0x7fa2d803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/532999776 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:51.143+0000 7fa2ef7fe700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 0x7fa2d803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:52.744+0000 7fa2ef7fe700 1 -- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 msgr2=0x7fa2d803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/532999776 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:52.744+0000 7fa2ef7fe700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 0x7fa2d803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:54.043+0000 7fa2ed7fa700 1 -- 172.21.15.132:0/4036961442 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fa2e00193d0 con 0x7fa2f0108480 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:54.043+0000 7fa2ed7fa700 1 -- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 msgr2=0x7fa2d803bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:54.043+0000 7fa2ed7fa700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 0x7fa2d803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.988 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.045+0000 7fa2ed7fa700 1 -- 172.21.15.132:0/4036961442 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fa2e002cb00 con 0x7fa2f0108480 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.045+0000 7fa2ed7fa700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa2d803cff0 0x7fa2d803f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.045+0000 7fa2ed7fa700 1 -- 172.21.15.132:0/4036961442 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa2e002ae50 con 0x7fa2d803cff0 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.046+0000 7fa2ef7fe700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa2d803cff0 0x7fa2d803f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.046+0000 7fa2ef7fe700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa2d803cff0 0x7fa2d803f3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4003a10 tx=0x7fa2e4009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.047+0000 7fa2ed7fa700 1 -- 172.21.15.132:0/4036961442 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fa2e002ae50 con 0x7fa2d803cff0 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.054+0000 7fa2f63cb700 1 -- 172.21.15.132:0/4036961442 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa2f0069b80 con 0x7fa2d803cff0 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.054+0000 7fa2ed7fa700 1 -- 172.21.15.132:0/4036961442 <== mgr.14152 v2:172.21.15.132:6800/636352941 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fa2f0069b80 con 0x7fa2d803cff0 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.054+0000 7fa2f63cb700 1 -- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa2d803cff0 msgr2=0x7fa2d803f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.054+0000 7fa2f63cb700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa2d803cff0 0x7fa2d803f3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4003a10 tx=0x7fa2e4009270 comp rx=0 tx=0).stop 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.055+0000 7fa2f63cb700 1 -- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 msgr2=0x7fa2f019c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.055+0000 7fa2f63cb700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 0x7fa2f019c360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0000c00 tx=0x7fa2e0011aa0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.055+0000 7fa2f63cb700 1 -- 172.21.15.132:0/4036961442 shutdown_connections 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.055+0000 7fa2f63cb700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa2d803cff0 0x7fa2d803f3e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.989 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.055+0000 7fa2f63cb700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f0108480 0x7fa2f019c360 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.990 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.055+0000 7fa2f63cb700 1 --2- 172.21.15.132:0/4036961442 >> [v2:172.21.15.132:6800/532999776,v1:172.21.15.132:6801/532999776] conn(0x7fa2d8039660 0x7fa2d803bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.990 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.055+0000 7fa2f63cb700 1 -- 172.21.15.132:0/4036961442 >> 172.21.15.132:0/4036961442 conn(0x7fa2f0103ac0 msgr2=0x7fa2f0104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:55.990 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.055+0000 7fa2f63cb700 1 -- 172.21.15.132:0/4036961442 shutdown_connections 2024-04-23T18:26:55.990 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:55.055+0000 7fa2f63cb700 1 -- 172.21.15.132:0/4036961442 wait complete. 2024-04-23T18:26:55.990 INFO:teuthology.orchestra.run.smithi132.stdout:mgr epoch 9 is available 2024-04-23T18:26:55.990 INFO:teuthology.orchestra.run.smithi132.stdout:Generating a dashboard self-signed certificate... 2024-04-23T18:26:56.230 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:56 smithi132 ceph-mon[29178]: [23/Apr/2024:18:26:54] ENGINE Bus STARTING 2024-04-23T18:26:56.230 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:56 smithi132 ceph-mon[29178]: [23/Apr/2024:18:26:54] ENGINE Serving on http://172.21.15.132:8765 2024-04-23T18:26:56.230 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:56 smithi132 ceph-mon[29178]: [23/Apr/2024:18:26:54] ENGINE Serving on https://172.21.15.132:7150 2024-04-23T18:26:56.230 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:56 smithi132 ceph-mon[29178]: [23/Apr/2024:18:26:54] ENGINE Bus STARTED 2024-04-23T18:26:56.230 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:56 smithi132 ceph-mon[29178]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T18:26:56.230 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:56 smithi132 ceph-mon[29178]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T18:26:56.230 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.443+0000 7ff61082d700 1 Processor -- start 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.443+0000 7ff61082d700 1 -- start start 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.443+0000 7ff61082d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 0x7ff60c1088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.443+0000 7ff61082d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c108e90 con 0x7ff60c1084a0 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.444+0000 7ff60a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 0x7ff60c1088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.444+0000 7ff60a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 0x7ff60c1088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55362/0 (socket says 172.21.15.132:55362) 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.444+0000 7ff60a59c700 1 -- 172.21.15.132:0/3760292956 learned_addr learned my addr 172.21.15.132:0/3760292956 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.444+0000 7ff60a59c700 1 -- 172.21.15.132:0/3760292956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c109650 con 0x7ff60c1084a0 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.444+0000 7ff60a59c700 1 --2- 172.21.15.132:0/3760292956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 0x7ff60c1088c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0098a0 tx=0x7ff5fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fa805d3e1c71f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.444+0000 7ff60959a700 1 -- 172.21.15.132:0/3760292956 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5fc004160 con 0x7ff60c1084a0 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.444+0000 7ff60959a700 1 -- 172.21.15.132:0/3760292956 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5fc004300 con 0x7ff60c1084a0 2024-04-23T18:26:57.551 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.444+0000 7ff60959a700 1 -- 172.21.15.132:0/3760292956 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5fc011a20 con 0x7ff60c1084a0 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.445+0000 7ff61082d700 1 -- 172.21.15.132:0/3760292956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 msgr2=0x7ff60c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.445+0000 7ff61082d700 1 --2- 172.21.15.132:0/3760292956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 0x7ff60c1088c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0098a0 tx=0x7ff5fc009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.445+0000 7ff61082d700 1 -- 172.21.15.132:0/3760292956 shutdown_connections 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.445+0000 7ff61082d700 1 --2- 172.21.15.132:0/3760292956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 0x7ff60c1088c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.445+0000 7ff61082d700 1 -- 172.21.15.132:0/3760292956 >> 172.21.15.132:0/3760292956 conn(0x7ff60c103ac0 msgr2=0x7ff60c105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.445+0000 7ff61082d700 1 -- 172.21.15.132:0/3760292956 shutdown_connections 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.445+0000 7ff61082d700 1 -- 172.21.15.132:0/3760292956 wait complete. 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.446+0000 7ff61082d700 1 Processor -- start 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.446+0000 7ff61082d700 1 -- start start 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.446+0000 7ff61082d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 0x7ff60c07db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.446+0000 7ff61082d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c10ca00 con 0x7ff60c1084a0 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.446+0000 7ff60a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 0x7ff60c07db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.446+0000 7ff60a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 0x7ff60c07db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55374/0 (socket says 172.21.15.132:55374) 2024-04-23T18:26:57.552 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.446+0000 7ff60a59c700 1 -- 172.21.15.132:0/3266862346 learned_addr learned my addr 172.21.15.132:0/3266862346 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.446+0000 7ff60a59c700 1 -- 172.21.15.132:0/3266862346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c07e0d0 con 0x7ff60c1084a0 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.446+0000 7ff60a59c700 1 --2- 172.21.15.132:0/3266862346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 0x7ff60c07db90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0119f0 tx=0x7ff5fc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.447+0000 7ff5fb7fe700 1 -- 172.21.15.132:0/3266862346 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5fc0039e0 con 0x7ff60c1084a0 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.447+0000 7ff61082d700 1 -- 172.21.15.132:0/3266862346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60c07c2c0 con 0x7ff60c1084a0 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.447+0000 7ff5fb7fe700 1 -- 172.21.15.132:0/3266862346 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5fc01b8f0 con 0x7ff60c1084a0 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.447+0000 7ff5fb7fe700 1 -- 172.21.15.132:0/3266862346 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5fc0237a0 con 0x7ff60c1084a0 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.447+0000 7ff61082d700 1 -- 172.21.15.132:0/3266862346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60c07c800 con 0x7ff60c1084a0 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.447+0000 7ff5fb7fe700 1 -- 172.21.15.132:0/3266862346 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7ff5fc02c3f0 con 0x7ff60c1084a0 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.448+0000 7ff61082d700 1 -- 172.21.15.132:0/3266862346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60c0640a0 con 0x7ff60c1084a0 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.448+0000 7ff5fb7fe700 1 --2- 172.21.15.132:0/3266862346 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff5f40394a0 0x7ff5f403b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.448+0000 7ff5fb7fe700 1 -- 172.21.15.132:0/3266862346 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff5fc04de20 con 0x7ff60c1084a0 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.448+0000 7ff609d9b700 1 --2- 172.21.15.132:0/3266862346 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff5f40394a0 0x7ff5f403b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.448+0000 7ff609d9b700 1 --2- 172.21.15.132:0/3266862346 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff5f40394a0 0x7ff5f403b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff600006fd0 tx=0x7ff600006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:57.553 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.453+0000 7ff5fb7fe700 1 -- 172.21.15.132:0/3266862346 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5fc01f030 con 0x7ff60c1084a0 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.649+0000 7ff5fb7fe700 1 -- 172.21.15.132:0/3266862346 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7ff5fc0193d0 con 0x7ff60c1084a0 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.710+0000 7ff61082d700 1 -- 172.21.15.132:0/3266862346 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7ff60c07d110 con 0x7ff5f40394a0 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.903+0000 7ff5fb7fe700 1 -- 172.21.15.132:0/3266862346 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7ff60c07d110 con 0x7ff5f40394a0 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.905+0000 7ff61082d700 1 -- 172.21.15.132:0/3266862346 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff5f40394a0 msgr2=0x7ff5f403b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.905+0000 7ff61082d700 1 --2- 172.21.15.132:0/3266862346 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff5f40394a0 0x7ff5f403b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff600006fd0 tx=0x7ff600006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.905+0000 7ff61082d700 1 -- 172.21.15.132:0/3266862346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 msgr2=0x7ff60c07db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.905+0000 7ff61082d700 1 --2- 172.21.15.132:0/3266862346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 0x7ff60c07db90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0119f0 tx=0x7ff5fc011ad0 comp rx=0 tx=0).stop 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.905+0000 7ff61082d700 1 -- 172.21.15.132:0/3266862346 shutdown_connections 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.905+0000 7ff61082d700 1 --2- 172.21.15.132:0/3266862346 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff5f40394a0 0x7ff5f403b960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.905+0000 7ff61082d700 1 --2- 172.21.15.132:0/3266862346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff60c1084a0 0x7ff60c07db90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.905+0000 7ff61082d700 1 -- 172.21.15.132:0/3266862346 >> 172.21.15.132:0/3266862346 conn(0x7ff60c103ac0 msgr2=0x7ff60c104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.906+0000 7ff61082d700 1 -- 172.21.15.132:0/3266862346 shutdown_connections 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:56.906+0000 7ff61082d700 1 -- 172.21.15.132:0/3266862346 wait complete. 2024-04-23T18:26:57.554 INFO:teuthology.orchestra.run.smithi132.stdout:Creating initial admin user... 2024-04-23T18:26:57.837 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:57 smithi132 ceph-mon[29178]: mgrmap e12: x(active, since 2s) 2024-04-23T18:26:57.837 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:57 smithi132 ceph-mon[29178]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:57.837 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:26:57.837 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:26:57.837 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:26:58.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:58 smithi132 ceph-mon[29178]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:58.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:26:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:26:59.654 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$KBrHBYLEO5gWFfvHQRg9/uCG19BTXLZeqFoPIO1iFkervrgQeBe8O", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713896818, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-23T18:26:59.654 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.946+0000 7f9a22274700 1 Processor -- start 2024-04-23T18:26:59.654 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.946+0000 7f9a22274700 1 -- start start 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.947+0000 7f9a22274700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 0x7f9a1c07c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.947+0000 7f9a22274700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c07c910 con 0x7f9a1c07de90 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.947+0000 7f9a1bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 0x7f9a1c07c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.947+0000 7f9a1bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 0x7f9a1c07c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55382/0 (socket says 172.21.15.132:55382) 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.947+0000 7f9a1bfff700 1 -- 172.21.15.132:0/446701739 learned_addr learned my addr 172.21.15.132:0/446701739 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.947+0000 7f9a1bfff700 1 -- 172.21.15.132:0/446701739 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c07ca90 con 0x7f9a1c07de90 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.947+0000 7f9a1bfff700 1 --2- 172.21.15.132:0/446701739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 0x7f9a1c07c340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0098a0 tx=0x7f9a0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b76d1471d712787 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.947+0000 7f9a1affd700 1 -- 172.21.15.132:0/446701739 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a0c004160 con 0x7f9a1c07de90 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.947+0000 7f9a1affd700 1 -- 172.21.15.132:0/446701739 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a0c004300 con 0x7f9a1c07de90 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.947+0000 7f9a1affd700 1 -- 172.21.15.132:0/446701739 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a0c011a20 con 0x7f9a1c07de90 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.948+0000 7f9a22274700 1 -- 172.21.15.132:0/446701739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 msgr2=0x7f9a1c07c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.948+0000 7f9a22274700 1 --2- 172.21.15.132:0/446701739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 0x7f9a1c07c340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0098a0 tx=0x7f9a0c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.948+0000 7f9a22274700 1 -- 172.21.15.132:0/446701739 shutdown_connections 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.948+0000 7f9a22274700 1 --2- 172.21.15.132:0/446701739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 0x7f9a1c07c340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:59.655 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.948+0000 7f9a22274700 1 -- 172.21.15.132:0/446701739 >> 172.21.15.132:0/446701739 conn(0x7f9a1c1039f0 msgr2=0x7f9a1c105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.948+0000 7f9a22274700 1 -- 172.21.15.132:0/446701739 shutdown_connections 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.948+0000 7f9a22274700 1 -- 172.21.15.132:0/446701739 wait complete. 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.949+0000 7f9a22274700 1 Processor -- start 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.949+0000 7f9a22274700 1 -- start start 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.949+0000 7f9a22274700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 0x7f9a1c19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.949+0000 7f9a22274700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c10e990 con 0x7f9a1c07de90 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.949+0000 7f9a1bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 0x7f9a1c19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.949+0000 7f9a1bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 0x7f9a1c19e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55388/0 (socket says 172.21.15.132:55388) 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.949+0000 7f9a1bfff700 1 -- 172.21.15.132:0/3578934658 learned_addr learned my addr 172.21.15.132:0/3578934658 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.949+0000 7f9a1bfff700 1 -- 172.21.15.132:0/3578934658 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c19ea20 con 0x7f9a1c07de90 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.949+0000 7f9a1bfff700 1 --2- 172.21.15.132:0/3578934658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 0x7f9a1c19e4e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c000c00 tx=0x7f9a0c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.950+0000 7f9a197fa700 1 -- 172.21.15.132:0/3578934658 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a0c003aa0 con 0x7f9a1c07de90 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.950+0000 7f9a197fa700 1 -- 172.21.15.132:0/3578934658 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a0c01b8f0 con 0x7f9a1c07de90 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.950+0000 7f9a22274700 1 -- 172.21.15.132:0/3578934658 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1c19ecb0 con 0x7f9a1c07de90 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.950+0000 7f9a197fa700 1 -- 172.21.15.132:0/3578934658 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a0c023870 con 0x7f9a1c07de90 2024-04-23T18:26:59.656 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.950+0000 7f9a22274700 1 -- 172.21.15.132:0/3578934658 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1c1a19a0 con 0x7f9a1c07de90 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.950+0000 7f9a197fa700 1 -- 172.21.15.132:0/3578934658 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f9a0c023b70 con 0x7f9a1c07de90 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.951+0000 7f9a197fa700 1 --2- 172.21.15.132:0/3578934658 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9a04039600 0x7f9a0403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.951+0000 7f9a197fa700 1 -- 172.21.15.132:0/3578934658 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9a0c04e030 con 0x7f9a1c07de90 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.951+0000 7f9a22274700 1 -- 172.21.15.132:0/3578934658 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1c0640a0 con 0x7f9a1c07de90 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.951+0000 7f9a1b7fe700 1 --2- 172.21.15.132:0/3578934658 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9a04039600 0x7f9a0403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.951+0000 7f9a1b7fe700 1 --2- 172.21.15.132:0/3578934658 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9a04039600 0x7f9a0403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a10006fd0 tx=0x7f9a10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:57.956+0000 7f9a197fa700 1 -- 172.21.15.132:0/3578934658 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a0c01f030 con 0x7f9a1c07de90 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:58.197+0000 7f9a22274700 1 -- 172.21.15.132:0/3578934658 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f9a1c19f190 con 0x7f9a04039600 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:58.421+0000 7f9a197fa700 1 -- 172.21.15.132:0/3578934658 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f9a1c19f190 con 0x7f9a04039600 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:58.423+0000 7f9a22274700 1 -- 172.21.15.132:0/3578934658 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9a04039600 msgr2=0x7f9a0403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:58.423+0000 7f9a22274700 1 --2- 172.21.15.132:0/3578934658 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9a04039600 0x7f9a0403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a10006fd0 tx=0x7f9a10006e40 comp rx=0 tx=0).stop 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:58.423+0000 7f9a22274700 1 -- 172.21.15.132:0/3578934658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 msgr2=0x7f9a1c19e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:58.423+0000 7f9a22274700 1 --2- 172.21.15.132:0/3578934658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 0x7f9a1c19e4e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c000c00 tx=0x7f9a0c011aa0 comp rx=0 tx=0).stop 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:58.423+0000 7f9a22274700 1 -- 172.21.15.132:0/3578934658 shutdown_connections 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:58.423+0000 7f9a22274700 1 --2- 172.21.15.132:0/3578934658 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9a04039600 0x7f9a0403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:59.657 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:58.423+0000 7f9a22274700 1 --2- 172.21.15.132:0/3578934658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c07de90 0x7f9a1c19e4e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:59.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:58.423+0000 7f9a22274700 1 -- 172.21.15.132:0/3578934658 >> 172.21.15.132:0/3578934658 conn(0x7f9a1c1039f0 msgr2=0x7f9a1c104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:59.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:58.423+0000 7f9a22274700 1 -- 172.21.15.132:0/3578934658 shutdown_connections 2024-04-23T18:26:59.658 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:26:58.423+0000 7f9a22274700 1 -- 172.21.15.132:0/3578934658 wait complete. 2024-04-23T18:26:59.658 INFO:teuthology.orchestra.run.smithi132.stdout:Fetching dashboard port number... 2024-04-23T18:27:00.836 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 8443 2024-04-23T18:27:00.836 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.065+0000 7fa702a4b700 1 Processor -- start 2024-04-23T18:27:00.836 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.065+0000 7fa702a4b700 1 -- start start 2024-04-23T18:27:00.836 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.065+0000 7fa702a4b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 0x7fa6fc108820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:00.836 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.065+0000 7fa702a4b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc108df0 con 0x7fa6fc108400 2024-04-23T18:27:00.836 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.066+0000 7fa6fbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 0x7fa6fc108820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.066+0000 7fa6fbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 0x7fa6fc108820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53498/0 (socket says 172.21.15.132:53498) 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.066+0000 7fa6fbfff700 1 -- 172.21.15.132:0/2576698253 learned_addr learned my addr 172.21.15.132:0/2576698253 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.066+0000 7fa6fbfff700 1 -- 172.21.15.132:0/2576698253 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc1095b0 con 0x7fa6fc108400 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.066+0000 7fa6fbfff700 1 --2- 172.21.15.132:0/2576698253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 0x7fa6fc108820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0098a0 tx=0x7fa6ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb544596c5fc3696 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.066+0000 7fa6faffd700 1 -- 172.21.15.132:0/2576698253 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6ec004160 con 0x7fa6fc108400 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.066+0000 7fa6faffd700 1 -- 172.21.15.132:0/2576698253 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6ec004300 con 0x7fa6fc108400 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.066+0000 7fa6faffd700 1 -- 172.21.15.132:0/2576698253 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6ec011a20 con 0x7fa6fc108400 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.067+0000 7fa702a4b700 1 -- 172.21.15.132:0/2576698253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 msgr2=0x7fa6fc108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.067+0000 7fa702a4b700 1 --2- 172.21.15.132:0/2576698253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 0x7fa6fc108820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0098a0 tx=0x7fa6ec009bb0 comp rx=0 tx=0).stop 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.067+0000 7fa702a4b700 1 -- 172.21.15.132:0/2576698253 shutdown_connections 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.067+0000 7fa702a4b700 1 --2- 172.21.15.132:0/2576698253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 0x7fa6fc108820 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.067+0000 7fa702a4b700 1 -- 172.21.15.132:0/2576698253 >> 172.21.15.132:0/2576698253 conn(0x7fa6fc1039e0 msgr2=0x7fa6fc105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.067+0000 7fa702a4b700 1 -- 172.21.15.132:0/2576698253 shutdown_connections 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.067+0000 7fa702a4b700 1 -- 172.21.15.132:0/2576698253 wait complete. 2024-04-23T18:27:00.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.067+0000 7fa702a4b700 1 Processor -- start 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.067+0000 7fa702a4b700 1 -- start start 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.068+0000 7fa702a4b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 0x7fa6fc19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.068+0000 7fa702a4b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc10c960 con 0x7fa6fc108400 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.068+0000 7fa6fbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 0x7fa6fc19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.068+0000 7fa6fbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 0x7fa6fc19c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53504/0 (socket says 172.21.15.132:53504) 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.068+0000 7fa6fbfff700 1 -- 172.21.15.132:0/3414886363 learned_addr learned my addr 172.21.15.132:0/3414886363 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.068+0000 7fa6fbfff700 1 -- 172.21.15.132:0/3414886363 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc19c810 con 0x7fa6fc108400 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.068+0000 7fa6fbfff700 1 --2- 172.21.15.132:0/3414886363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 0x7fa6fc19c2d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009520 tx=0x7fa6ec011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.068+0000 7fa6f97fa700 1 -- 172.21.15.132:0/3414886363 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6ec0039e0 con 0x7fa6fc108400 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.068+0000 7fa6f97fa700 1 -- 172.21.15.132:0/3414886363 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6ec01b8f0 con 0x7fa6fc108400 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.068+0000 7fa6f97fa700 1 -- 172.21.15.132:0/3414886363 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6ec019770 con 0x7fa6fc108400 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.068+0000 7fa702a4b700 1 -- 172.21.15.132:0/3414886363 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6fc19caa0 con 0x7fa6fc108400 2024-04-23T18:27:00.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.068+0000 7fa702a4b700 1 -- 172.21.15.132:0/3414886363 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6fc19f790 con 0x7fa6fc108400 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.069+0000 7fa6f97fa700 1 -- 172.21.15.132:0/3414886363 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fa6ec01b440 con 0x7fa6fc108400 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.069+0000 7fa702a4b700 1 -- 172.21.15.132:0/3414886363 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6fc0640a0 con 0x7fa6fc108400 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.069+0000 7fa6f97fa700 1 --2- 172.21.15.132:0/3414886363 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6e4039650 0x7fa6e403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.070+0000 7fa6f97fa700 1 -- 172.21.15.132:0/3414886363 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa6ec04dfb0 con 0x7fa6fc108400 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.070+0000 7fa6fb7fe700 1 --2- 172.21.15.132:0/3414886363 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6e4039650 0x7fa6e403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.070+0000 7fa6fb7fe700 1 --2- 172.21.15.132:0/3414886363 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6e4039650 0x7fa6e403bb10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0006fd0 tx=0x7fa6f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.075+0000 7fa6f97fa700 1 -- 172.21.15.132:0/3414886363 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6ec01f030 con 0x7fa6fc108400 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.096+0000 7fa6f97fa700 1 -- 172.21.15.132:0/3414886363 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fa6ec010380 con 0x7fa6fc108400 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.277+0000 7fa702a4b700 1 -- 172.21.15.132:0/3414886363 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa6fc19f960 con 0x7fa6fc108400 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.277+0000 7fa6f97fa700 1 -- 172.21.15.132:0/3414886363 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa6ec052690 con 0x7fa6fc108400 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.280+0000 7fa702a4b700 1 -- 172.21.15.132:0/3414886363 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6e4039650 msgr2=0x7fa6e403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.280+0000 7fa702a4b700 1 --2- 172.21.15.132:0/3414886363 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6e4039650 0x7fa6e403bb10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0006fd0 tx=0x7fa6f0006e40 comp rx=0 tx=0).stop 2024-04-23T18:27:00.839 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.280+0000 7fa702a4b700 1 -- 172.21.15.132:0/3414886363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 msgr2=0x7fa6fc19c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:00.840 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.280+0000 7fa702a4b700 1 --2- 172.21.15.132:0/3414886363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 0x7fa6fc19c2d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009520 tx=0x7fa6ec011aa0 comp rx=0 tx=0).stop 2024-04-23T18:27:00.840 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.280+0000 7fa702a4b700 1 -- 172.21.15.132:0/3414886363 shutdown_connections 2024-04-23T18:27:00.840 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.280+0000 7fa702a4b700 1 --2- 172.21.15.132:0/3414886363 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6e4039650 0x7fa6e403bb10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:00.840 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.280+0000 7fa702a4b700 1 --2- 172.21.15.132:0/3414886363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6fc108400 0x7fa6fc19c2d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:00.840 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.280+0000 7fa702a4b700 1 -- 172.21.15.132:0/3414886363 >> 172.21.15.132:0/3414886363 conn(0x7fa6fc1039e0 msgr2=0x7fa6fc104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:00.840 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.280+0000 7fa702a4b700 1 -- 172.21.15.132:0/3414886363 shutdown_connections 2024-04-23T18:27:00.840 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:00.280+0000 7fa702a4b700 1 -- 172.21.15.132:0/3414886363 wait complete. 2024-04-23T18:27:00.843 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:27:00.843 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout disabled 2024-04-23T18:27:00.850 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:27:00.850 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout inactive 2024-04-23T18:27:00.850 INFO:teuthology.orchestra.run.smithi132.stdout:firewalld.service is not enabled 2024-04-23T18:27:00.850 INFO:teuthology.orchestra.run.smithi132.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-23T18:27:00.851 INFO:teuthology.orchestra.run.smithi132.stdout:Ceph Dashboard is now available at: 2024-04-23T18:27:00.852 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:27:00.852 INFO:teuthology.orchestra.run.smithi132.stdout: URL: https://smithi132:8443/ 2024-04-23T18:27:00.852 INFO:teuthology.orchestra.run.smithi132.stdout: User: admin 2024-04-23T18:27:00.852 INFO:teuthology.orchestra.run.smithi132.stdout: Password: xb2bcpaljv 2024-04-23T18:27:00.852 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:27:00.852 INFO:teuthology.orchestra.run.smithi132.stdout:Saving cluster configuration to /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config directory 2024-04-23T18:27:00.852 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling autotune for osd_memory_target 2024-04-23T18:27:01.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:01 smithi132 ceph-mon[29178]: mgrmap e13: x(active, since 6s) 2024-04-23T18:27:01.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:01 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3414886363' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-23T18:27:02.084 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.295+0000 7f303ca7f700 1 Processor -- start 2024-04-23T18:27:02.084 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.295+0000 7f303ca7f700 1 -- start start 2024-04-23T18:27:02.084 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.295+0000 7f303ca7f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 0x7f3038108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:02.084 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.295+0000 7f303ca7f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3038108e60 con 0x7f3038108470 2024-04-23T18:27:02.084 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.295+0000 7f303659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 0x7f3038108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:02.084 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.295+0000 7f303659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 0x7f3038108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53506/0 (socket says 172.21.15.132:53506) 2024-04-23T18:27:02.084 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.295+0000 7f303659c700 1 -- 172.21.15.132:0/1314842798 learned_addr learned my addr 172.21.15.132:0/1314842798 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.296+0000 7f303659c700 1 -- 172.21.15.132:0/1314842798 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3038109620 con 0x7f3038108470 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.296+0000 7f303659c700 1 --2- 172.21.15.132:0/1314842798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 0x7f3038108890 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f30280098a0 tx=0x7f3028009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc44dbf5a879456b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.296+0000 7f3035d9b700 1 -- 172.21.15.132:0/1314842798 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3028004160 con 0x7f3038108470 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.296+0000 7f3035d9b700 1 -- 172.21.15.132:0/1314842798 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3028004300 con 0x7f3038108470 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.296+0000 7f3035d9b700 1 -- 172.21.15.132:0/1314842798 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3028011a20 con 0x7f3038108470 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.296+0000 7f303ca7f700 1 -- 172.21.15.132:0/1314842798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 msgr2=0x7f3038108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.296+0000 7f303ca7f700 1 --2- 172.21.15.132:0/1314842798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 0x7f3038108890 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f30280098a0 tx=0x7f3028009bb0 comp rx=0 tx=0).stop 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.297+0000 7f303ca7f700 1 -- 172.21.15.132:0/1314842798 shutdown_connections 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.297+0000 7f303ca7f700 1 --2- 172.21.15.132:0/1314842798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 0x7f3038108890 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.297+0000 7f303ca7f700 1 -- 172.21.15.132:0/1314842798 >> 172.21.15.132:0/1314842798 conn(0x7f3038103a50 msgr2=0x7f3038105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.297+0000 7f303ca7f700 1 -- 172.21.15.132:0/1314842798 shutdown_connections 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.297+0000 7f303ca7f700 1 -- 172.21.15.132:0/1314842798 wait complete. 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.297+0000 7f303ca7f700 1 Processor -- start 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.297+0000 7f303ca7f700 1 -- start start 2024-04-23T18:27:02.085 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.298+0000 7f303ca7f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 0x7f303807db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.298+0000 7f303ca7f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f303810c9d0 con 0x7f3038108470 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.298+0000 7f303659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 0x7f303807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.298+0000 7f303659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 0x7f303807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53508/0 (socket says 172.21.15.132:53508) 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.298+0000 7f303659c700 1 -- 172.21.15.132:0/3713797523 learned_addr learned my addr 172.21.15.132:0/3713797523 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.298+0000 7f303659c700 1 -- 172.21.15.132:0/3713797523 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303807e0d0 con 0x7f3038108470 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.298+0000 7f303659c700 1 --2- 172.21.15.132:0/3713797523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 0x7f303807db90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3028000c00 tx=0x7f3028011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.298+0000 7f302ffff700 1 -- 172.21.15.132:0/3713797523 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3028003aa0 con 0x7f3038108470 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.298+0000 7f302ffff700 1 -- 172.21.15.132:0/3713797523 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f302801b8f0 con 0x7f3038108470 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.298+0000 7f303ca7f700 1 -- 172.21.15.132:0/3713797523 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303807c2c0 con 0x7f3038108470 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.298+0000 7f302ffff700 1 -- 172.21.15.132:0/3713797523 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3028023890 con 0x7f3038108470 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.298+0000 7f303ca7f700 1 -- 172.21.15.132:0/3713797523 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303807c800 con 0x7f3038108470 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.299+0000 7f302ffff700 1 -- 172.21.15.132:0/3713797523 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f302802c3f0 con 0x7f3038108470 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.299+0000 7f303ca7f700 1 -- 172.21.15.132:0/3713797523 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30380640a0 con 0x7f3038108470 2024-04-23T18:27:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.299+0000 7f302ffff700 1 --2- 172.21.15.132:0/3713797523 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f30240396a0 0x7f302403bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.299+0000 7f302ffff700 1 -- 172.21.15.132:0/3713797523 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f302804e0a0 con 0x7f3038108470 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.299+0000 7f302dd9b700 1 --2- 172.21.15.132:0/3713797523 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f30240396a0 0x7f302403bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.300+0000 7f302dd9b700 1 --2- 172.21.15.132:0/3713797523 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f30240396a0 0x7f302403bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3020006fd0 tx=0x7f3020006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.304+0000 7f302ffff700 1 -- 172.21.15.132:0/3713797523 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f302801f030 con 0x7f3038108470 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.507+0000 7f303ca7f700 1 -- 172.21.15.132:0/3713797523 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f303807d150 con 0x7f3038108470 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.507+0000 7f302ffff700 1 -- 172.21.15.132:0/3713797523 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f303807d150 con 0x7f3038108470 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.509+0000 7f303ca7f700 1 -- 172.21.15.132:0/3713797523 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f30240396a0 msgr2=0x7f302403bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.509+0000 7f303ca7f700 1 --2- 172.21.15.132:0/3713797523 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f30240396a0 0x7f302403bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3020006fd0 tx=0x7f3020006e40 comp rx=0 tx=0).stop 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.509+0000 7f303ca7f700 1 -- 172.21.15.132:0/3713797523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 msgr2=0x7f303807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.509+0000 7f303ca7f700 1 --2- 172.21.15.132:0/3713797523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 0x7f303807db90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3028000c00 tx=0x7f3028011aa0 comp rx=0 tx=0).stop 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.509+0000 7f303ca7f700 1 -- 172.21.15.132:0/3713797523 shutdown_connections 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.509+0000 7f303ca7f700 1 --2- 172.21.15.132:0/3713797523 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f30240396a0 0x7f302403bb60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.509+0000 7f303ca7f700 1 --2- 172.21.15.132:0/3713797523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3038108470 0x7f303807db90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.509+0000 7f303ca7f700 1 -- 172.21.15.132:0/3713797523 >> 172.21.15.132:0/3713797523 conn(0x7f3038103a50 msgr2=0x7f3038104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:02.087 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.509+0000 7f303ca7f700 1 -- 172.21.15.132:0/3713797523 shutdown_connections 2024-04-23T18:27:02.088 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:01.509+0000 7f303ca7f700 1 -- 172.21.15.132:0/3713797523 wait complete. 2024-04-23T18:27:03.382 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.511+0000 7f6ff4304700 1 Processor -- start 2024-04-23T18:27:03.382 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.511+0000 7f6ff4304700 1 -- start start 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.512+0000 7f6ff4304700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 0x7f6fec108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.512+0000 7f6ff4304700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec108e60 con 0x7f6fec108470 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.512+0000 7f6ff20a0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 0x7f6fec108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.512+0000 7f6ff20a0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 0x7f6fec108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53510/0 (socket says 172.21.15.132:53510) 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.512+0000 7f6ff20a0700 1 -- 172.21.15.132:0/419644309 learned_addr learned my addr 172.21.15.132:0/419644309 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.512+0000 7f6ff20a0700 1 -- 172.21.15.132:0/419644309 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec109620 con 0x7f6fec108470 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.512+0000 7f6ff20a0700 1 --2- 172.21.15.132:0/419644309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 0x7f6fec108890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0098a0 tx=0x7f6fdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb87181436988f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.512+0000 7f6ff109e700 1 -- 172.21.15.132:0/419644309 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fdc004160 con 0x7f6fec108470 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.512+0000 7f6ff109e700 1 -- 172.21.15.132:0/419644309 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fdc004300 con 0x7f6fec108470 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.512+0000 7f6ff109e700 1 -- 172.21.15.132:0/419644309 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fdc011a20 con 0x7f6fec108470 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.513+0000 7f6ff4304700 1 -- 172.21.15.132:0/419644309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 msgr2=0x7f6fec108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.513+0000 7f6ff4304700 1 --2- 172.21.15.132:0/419644309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 0x7f6fec108890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0098a0 tx=0x7f6fdc009bb0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.513+0000 7f6ff4304700 1 -- 172.21.15.132:0/419644309 shutdown_connections 2024-04-23T18:27:03.383 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.513+0000 7f6ff4304700 1 --2- 172.21.15.132:0/419644309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 0x7f6fec108890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.513+0000 7f6ff4304700 1 -- 172.21.15.132:0/419644309 >> 172.21.15.132:0/419644309 conn(0x7f6fec103a50 msgr2=0x7f6fec105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.513+0000 7f6ff4304700 1 -- 172.21.15.132:0/419644309 shutdown_connections 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.513+0000 7f6ff4304700 1 -- 172.21.15.132:0/419644309 wait complete. 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6ff4304700 1 Processor -- start 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6ff4304700 1 -- start start 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6ff4304700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 0x7f6fec19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6ff4304700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec10c9d0 con 0x7f6fec108470 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6ff20a0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 0x7f6fec19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6ff20a0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 0x7f6fec19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53516/0 (socket says 172.21.15.132:53516) 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6ff20a0700 1 -- 172.21.15.132:0/91141611 learned_addr learned my addr 172.21.15.132:0/91141611 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6ff20a0700 1 -- 172.21.15.132:0/91141611 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec19c880 con 0x7f6fec108470 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6ff20a0700 1 --2- 172.21.15.132:0/91141611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 0x7f6fec19c340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0119f0 tx=0x7f6fdc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6fe37fe700 1 -- 172.21.15.132:0/91141611 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fdc0039e0 con 0x7f6fec108470 2024-04-23T18:27:03.384 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6fe37fe700 1 -- 172.21.15.132:0/91141611 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fdc01b8f0 con 0x7f6fec108470 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6fe37fe700 1 -- 172.21.15.132:0/91141611 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fdc0237a0 con 0x7f6fec108470 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.514+0000 7f6ff4304700 1 -- 172.21.15.132:0/91141611 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fec19cb10 con 0x7f6fec108470 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.515+0000 7f6ff4304700 1 -- 172.21.15.132:0/91141611 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fec19f800 con 0x7f6fec108470 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.515+0000 7f6fe37fe700 1 -- 172.21.15.132:0/91141611 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f6fdc02c3f0 con 0x7f6fec108470 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.515+0000 7f6ff4304700 1 -- 172.21.15.132:0/91141611 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fec0640a0 con 0x7f6fec108470 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.515+0000 7f6fe37fe700 1 --2- 172.21.15.132:0/91141611 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6fd8039650 0x7f6fd803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.516+0000 7f6fe37fe700 1 -- 172.21.15.132:0/91141611 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6fdc04e210 con 0x7f6fec108470 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.516+0000 7f6ff189f700 1 --2- 172.21.15.132:0/91141611 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6fd8039650 0x7f6fd803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.516+0000 7f6ff189f700 1 --2- 172.21.15.132:0/91141611 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6fd8039650 0x7f6fd803bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8006fd0 tx=0x7f6fe8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.520+0000 7f6fe37fe700 1 -- 172.21.15.132:0/91141611 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6fdc01f030 con 0x7f6fec108470 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.830+0000 7f6ff4304700 1 -- 172.21.15.132:0/91141611 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f6fec19fdb0 con 0x7f6fec108470 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.833+0000 7f6fe37fe700 1 -- 172.21.15.132:0/91141611 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f6fdc0105e0 con 0x7f6fec108470 2024-04-23T18:27:03.385 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.835+0000 7f6ff4304700 1 -- 172.21.15.132:0/91141611 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6fd8039650 msgr2=0x7f6fd803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.835+0000 7f6ff4304700 1 --2- 172.21.15.132:0/91141611 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6fd8039650 0x7f6fd803bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8006fd0 tx=0x7f6fe8006e40 comp rx=0 tx=0).stop 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.835+0000 7f6ff4304700 1 -- 172.21.15.132:0/91141611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 msgr2=0x7f6fec19c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.835+0000 7f6ff4304700 1 --2- 172.21.15.132:0/91141611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 0x7f6fec19c340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0119f0 tx=0x7f6fdc011ad0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.835+0000 7f6ff4304700 1 -- 172.21.15.132:0/91141611 shutdown_connections 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.835+0000 7f6ff4304700 1 --2- 172.21.15.132:0/91141611 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6fd8039650 0x7f6fd803bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.835+0000 7f6ff4304700 1 --2- 172.21.15.132:0/91141611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fec108470 0x7f6fec19c340 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.835+0000 7f6ff4304700 1 -- 172.21.15.132:0/91141611 >> 172.21.15.132:0/91141611 conn(0x7f6fec103a50 msgr2=0x7f6fec104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.835+0000 7f6ff4304700 1 -- 172.21.15.132:0/91141611 shutdown_connections 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-04-23T18:27:02.835+0000 7f6ff4304700 1 -- 172.21.15.132:0/91141611 wait complete. 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:27:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout:Or, if you are only running a single cluster on this host: 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: ceph telemetry on 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout:For more information see: 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:27:03.387 INFO:teuthology.orchestra.run.smithi132.stdout:Bootstrap complete. 2024-04-23T18:27:03.416 INFO:tasks.cephadm:Fetching config... 2024-04-23T18:27:03.416 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:27:03.416 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-23T18:27:03.431 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-23T18:27:03.432 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:27:03.432 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-23T18:27:03.486 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-23T18:27:03.486 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:27:03.486 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-23T18:27:03.550 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-23T18:27:03.550 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:27:03.550 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-23T18:27:03.605 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-23T18:27:03.605 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHj4S72TxmaMvOqsIsUwYoMPFCypC7LiLyuAImJ3Idx1lHq0LsHVkjdKTxfM34kAmuDE6NxkcnKsbmqcpKnQB2cphZXRFL5zf6s/5gPpU8tqRaqrWVJsJHiFoSSp/ZJMigqbOQBqAQhkzewh8JFpgOr91UZKaWHvcQbojaoeJeEX7n6w9FbuDnn0OvU5lFoGiw9ElDuwbvyI184rA+TmBD8kDIDDmKfzKXf8TZuXt5RWOCUM4N4IUyjcZLXmDx4tE/0SEMAqXq/gNZuxM2DH2kIvEQOmAkfApK0H3C2hZ76RQ+kZ4z1j6PdKlWr2HgB5LSce/PBcr+2V3sdCtGjB1JeqcHrsWyZELl8TJkn7sEr5bsesO8YsRERT0qmlK+DzfTL7BWKDjcwlSP2PaEZ8LZL7Fn3FpFSE0buli5c0WPLpFfvXyi7zQaESt6+DPriDHXmERYA6a9HChfTum0FdZKOkRKBiJnYfQQEjDV6VqqwgMSplpgYdRIjvbZ+/lalVk= ceph-c34642b4-019e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T18:27:03.677 INFO:teuthology.orchestra.run.smithi132.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHj4S72TxmaMvOqsIsUwYoMPFCypC7LiLyuAImJ3Idx1lHq0LsHVkjdKTxfM34kAmuDE6NxkcnKsbmqcpKnQB2cphZXRFL5zf6s/5gPpU8tqRaqrWVJsJHiFoSSp/ZJMigqbOQBqAQhkzewh8JFpgOr91UZKaWHvcQbojaoeJeEX7n6w9FbuDnn0OvU5lFoGiw9ElDuwbvyI184rA+TmBD8kDIDDmKfzKXf8TZuXt5RWOCUM4N4IUyjcZLXmDx4tE/0SEMAqXq/gNZuxM2DH2kIvEQOmAkfApK0H3C2hZ76RQ+kZ4z1j6PdKlWr2HgB5LSce/PBcr+2V3sdCtGjB1JeqcHrsWyZELl8TJkn7sEr5bsesO8YsRERT0qmlK+DzfTL7BWKDjcwlSP2PaEZ8LZL7Fn3FpFSE0buli5c0WPLpFfvXyi7zQaESt6+DPriDHXmERYA6a9HChfTum0FdZKOkRKBiJnYfQQEjDV6VqqwgMSplpgYdRIjvbZ+/lalVk= ceph-c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:03.689 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHj4S72TxmaMvOqsIsUwYoMPFCypC7LiLyuAImJ3Idx1lHq0LsHVkjdKTxfM34kAmuDE6NxkcnKsbmqcpKnQB2cphZXRFL5zf6s/5gPpU8tqRaqrWVJsJHiFoSSp/ZJMigqbOQBqAQhkzewh8JFpgOr91UZKaWHvcQbojaoeJeEX7n6w9FbuDnn0OvU5lFoGiw9ElDuwbvyI184rA+TmBD8kDIDDmKfzKXf8TZuXt5RWOCUM4N4IUyjcZLXmDx4tE/0SEMAqXq/gNZuxM2DH2kIvEQOmAkfApK0H3C2hZ76RQ+kZ4z1j6PdKlWr2HgB5LSce/PBcr+2V3sdCtGjB1JeqcHrsWyZELl8TJkn7sEr5bsesO8YsRERT0qmlK+DzfTL7BWKDjcwlSP2PaEZ8LZL7Fn3FpFSE0buli5c0WPLpFfvXyi7zQaESt6+DPriDHXmERYA6a9HChfTum0FdZKOkRKBiJnYfQQEjDV6VqqwgMSplpgYdRIjvbZ+/lalVk= ceph-c34642b4-019e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T18:27:03.728 INFO:teuthology.orchestra.run.smithi148.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHj4S72TxmaMvOqsIsUwYoMPFCypC7LiLyuAImJ3Idx1lHq0LsHVkjdKTxfM34kAmuDE6NxkcnKsbmqcpKnQB2cphZXRFL5zf6s/5gPpU8tqRaqrWVJsJHiFoSSp/ZJMigqbOQBqAQhkzewh8JFpgOr91UZKaWHvcQbojaoeJeEX7n6w9FbuDnn0OvU5lFoGiw9ElDuwbvyI184rA+TmBD8kDIDDmKfzKXf8TZuXt5RWOCUM4N4IUyjcZLXmDx4tE/0SEMAqXq/gNZuxM2DH2kIvEQOmAkfApK0H3C2hZ76RQ+kZ4z1j6PdKlWr2HgB5LSce/PBcr+2V3sdCtGjB1JeqcHrsWyZELl8TJkn7sEr5bsesO8YsRERT0qmlK+DzfTL7BWKDjcwlSP2PaEZ8LZL7Fn3FpFSE0buli5c0WPLpFfvXyi7zQaESt6+DPriDHXmERYA6a9HChfTum0FdZKOkRKBiJnYfQQEjDV6VqqwgMSplpgYdRIjvbZ+/lalVk= ceph-c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:03.740 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHj4S72TxmaMvOqsIsUwYoMPFCypC7LiLyuAImJ3Idx1lHq0LsHVkjdKTxfM34kAmuDE6NxkcnKsbmqcpKnQB2cphZXRFL5zf6s/5gPpU8tqRaqrWVJsJHiFoSSp/ZJMigqbOQBqAQhkzewh8JFpgOr91UZKaWHvcQbojaoeJeEX7n6w9FbuDnn0OvU5lFoGiw9ElDuwbvyI184rA+TmBD8kDIDDmKfzKXf8TZuXt5RWOCUM4N4IUyjcZLXmDx4tE/0SEMAqXq/gNZuxM2DH2kIvEQOmAkfApK0H3C2hZ76RQ+kZ4z1j6PdKlWr2HgB5LSce/PBcr+2V3sdCtGjB1JeqcHrsWyZELl8TJkn7sEr5bsesO8YsRERT0qmlK+DzfTL7BWKDjcwlSP2PaEZ8LZL7Fn3FpFSE0buli5c0WPLpFfvXyi7zQaESt6+DPriDHXmERYA6a9HChfTum0FdZKOkRKBiJnYfQQEjDV6VqqwgMSplpgYdRIjvbZ+/lalVk= ceph-c34642b4-019e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T18:27:03.779 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHj4S72TxmaMvOqsIsUwYoMPFCypC7LiLyuAImJ3Idx1lHq0LsHVkjdKTxfM34kAmuDE6NxkcnKsbmqcpKnQB2cphZXRFL5zf6s/5gPpU8tqRaqrWVJsJHiFoSSp/ZJMigqbOQBqAQhkzewh8JFpgOr91UZKaWHvcQbojaoeJeEX7n6w9FbuDnn0OvU5lFoGiw9ElDuwbvyI184rA+TmBD8kDIDDmKfzKXf8TZuXt5RWOCUM4N4IUyjcZLXmDx4tE/0SEMAqXq/gNZuxM2DH2kIvEQOmAkfApK0H3C2hZ76RQ+kZ4z1j6PdKlWr2HgB5LSce/PBcr+2V3sdCtGjB1JeqcHrsWyZELl8TJkn7sEr5bsesO8YsRERT0qmlK+DzfTL7BWKDjcwlSP2PaEZ8LZL7Fn3FpFSE0buli5c0WPLpFfvXyi7zQaESt6+DPriDHXmERYA6a9HChfTum0FdZKOkRKBiJnYfQQEjDV6VqqwgMSplpgYdRIjvbZ+/lalVk= ceph-c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:03.792 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-23T18:27:04.038 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:27:04.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:03 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/91141611' entity='client.admin' 2024-04-23T18:27:06.661 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:06 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:06.661 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:06 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:06.661 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:06 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi132", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:27:06.661 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:06 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:06.661 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:06 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:06.661 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:06 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.675+0000 7f1eda316700 1 -- 172.21.15.132:0/2682698859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41024c0 msgr2=0x7f1ed41028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.675+0000 7f1eda316700 1 --2- 172.21.15.132:0/2682698859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41024c0 0x7f1ed41028e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009910 tx=0x7f1ec4009c20 comp rx=0 tx=0).stop 2024-04-23T18:27:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.675+0000 7f1eda316700 1 -- 172.21.15.132:0/2682698859 shutdown_connections 2024-04-23T18:27:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.675+0000 7f1eda316700 1 --2- 172.21.15.132:0/2682698859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41024c0 0x7f1ed41028e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.675+0000 7f1eda316700 1 -- 172.21.15.132:0/2682698859 >> 172.21.15.132:0/2682698859 conn(0x7f1ed40fda60 msgr2=0x7f1ed40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.675+0000 7f1eda316700 1 -- 172.21.15.132:0/2682698859 shutdown_connections 2024-04-23T18:27:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.675+0000 7f1eda316700 1 -- 172.21.15.132:0/2682698859 wait complete. 2024-04-23T18:27:06.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.676+0000 7f1eda316700 1 Processor -- start 2024-04-23T18:27:06.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.676+0000 7f1eda316700 1 -- start start 2024-04-23T18:27:06.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.676+0000 7f1eda316700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41024c0 0x7f1ed4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:06.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.676+0000 7f1eda316700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4108400 con 0x7f1ed41024c0 2024-04-23T18:27:06.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.676+0000 7f1ed3fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41024c0 0x7f1ed4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:06.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.676+0000 7f1ed3fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41024c0 0x7f1ed4197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53540/0 (socket says 172.21.15.132:53540) 2024-04-23T18:27:06.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.676+0000 7f1ed3fff700 1 -- 172.21.15.132:0/848866949 learned_addr learned my addr 172.21.15.132:0/848866949 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:06.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.676+0000 7f1ed3fff700 1 -- 172.21.15.132:0/848866949 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed41980a0 con 0x7f1ed41024c0 2024-04-23T18:27:06.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.677+0000 7f1ed3fff700 1 --2- 172.21.15.132:0/848866949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41024c0 0x7f1ed4197b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4005950 tx=0x7f1ec40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:06.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.677+0000 7f1ed17fa700 1 -- 172.21.15.132:0/848866949 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ec4026070 con 0x7f1ed41024c0 2024-04-23T18:27:06.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.677+0000 7f1eda316700 1 -- 172.21.15.132:0/848866949 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed4198330 con 0x7f1ed41024c0 2024-04-23T18:27:06.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.677+0000 7f1ed17fa700 1 -- 172.21.15.132:0/848866949 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec400bc00 con 0x7f1ed41024c0 2024-04-23T18:27:06.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.677+0000 7f1ed17fa700 1 -- 172.21.15.132:0/848866949 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ec4003a80 con 0x7f1ed41024c0 2024-04-23T18:27:06.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.677+0000 7f1eda316700 1 -- 172.21.15.132:0/848866949 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed419b020 con 0x7f1ed41024c0 2024-04-23T18:27:06.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.678+0000 7f1ed17fa700 1 -- 172.21.15.132:0/848866949 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f1ec4029070 con 0x7f1ed41024c0 2024-04-23T18:27:06.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.678+0000 7f1eda316700 1 -- 172.21.15.132:0/848866949 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed4108b80 con 0x7f1ed41024c0 2024-04-23T18:27:06.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.678+0000 7f1ed17fa700 1 --2- 172.21.15.132:0/848866949 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ebc039650 0x7f1ebc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:06.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.678+0000 7f1ed17fa700 1 -- 172.21.15.132:0/848866949 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1ec401e070 con 0x7f1ed41024c0 2024-04-23T18:27:06.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.678+0000 7f1ed37fe700 1 --2- 172.21.15.132:0/848866949 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ebc039650 0x7f1ebc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:06.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.678+0000 7f1ed37fe700 1 --2- 172.21.15.132:0/848866949 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ebc039650 0x7f1ebc03bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8006fd0 tx=0x7f1ec8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:06.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.683+0000 7f1ed17fa700 1 -- 172.21.15.132:0/848866949 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ec4013070 con 0x7f1ed41024c0 2024-04-23T18:27:06.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.889+0000 7f1eda316700 1 -- 172.21.15.132:0/848866949 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f1ed402e090 con 0x7f1ed41024c0 2024-04-23T18:27:06.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.891+0000 7f1ed17fa700 1 -- 172.21.15.132:0/848866949 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f1ec4030020 con 0x7f1ed41024c0 2024-04-23T18:27:06.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.894+0000 7f1eda316700 1 -- 172.21.15.132:0/848866949 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ebc039650 msgr2=0x7f1ebc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:06.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.894+0000 7f1eda316700 1 --2- 172.21.15.132:0/848866949 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ebc039650 0x7f1ebc03bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8006fd0 tx=0x7f1ec8006e40 comp rx=0 tx=0).stop 2024-04-23T18:27:06.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.894+0000 7f1eda316700 1 -- 172.21.15.132:0/848866949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41024c0 msgr2=0x7f1ed4197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:06.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.894+0000 7f1eda316700 1 --2- 172.21.15.132:0/848866949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41024c0 0x7f1ed4197b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4005950 tx=0x7f1ec40057f0 comp rx=0 tx=0).stop 2024-04-23T18:27:06.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.894+0000 7f1eda316700 1 -- 172.21.15.132:0/848866949 shutdown_connections 2024-04-23T18:27:06.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.894+0000 7f1eda316700 1 --2- 172.21.15.132:0/848866949 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ebc039650 0x7f1ebc03bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:06.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.894+0000 7f1eda316700 1 --2- 172.21.15.132:0/848866949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41024c0 0x7f1ed4197b60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:06.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.894+0000 7f1eda316700 1 -- 172.21.15.132:0/848866949 >> 172.21.15.132:0/848866949 conn(0x7f1ed40fda60 msgr2=0x7f1ed40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:06.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.895+0000 7f1eda316700 1 -- 172.21.15.132:0/848866949 shutdown_connections 2024-04-23T18:27:06.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:06.895+0000 7f1eda316700 1 -- 172.21.15.132:0/848866949 wait complete. 2024-04-23T18:27:07.759 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-23T18:27:07.759 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-23T18:27:08.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:07 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/848866949' entity='client.admin' 2024-04-23T18:27:08.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:07 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:08.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:07 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:08.211 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:27:09.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.672+0000 7f06fcf23700 1 -- 172.21.15.132:0/2254661066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f81013b0 msgr2=0x7f06f81017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:09.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.672+0000 7f06fcf23700 1 --2- 172.21.15.132:0/2254661066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f81013b0 0x7f06f81017d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009910 tx=0x7f06e8009c20 comp rx=0 tx=0).stop 2024-04-23T18:27:09.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.672+0000 7f06fcf23700 1 -- 172.21.15.132:0/2254661066 shutdown_connections 2024-04-23T18:27:09.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.672+0000 7f06fcf23700 1 --2- 172.21.15.132:0/2254661066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f81013b0 0x7f06f81017d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:09.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.672+0000 7f06fcf23700 1 -- 172.21.15.132:0/2254661066 >> 172.21.15.132:0/2254661066 conn(0x7f06f80fc950 msgr2=0x7f06f80fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:09.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.672+0000 7f06fcf23700 1 -- 172.21.15.132:0/2254661066 shutdown_connections 2024-04-23T18:27:09.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.672+0000 7f06fcf23700 1 -- 172.21.15.132:0/2254661066 wait complete. 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.673+0000 7f06fcf23700 1 Processor -- start 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.673+0000 7f06fcf23700 1 -- start start 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.673+0000 7f06fcf23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f81013b0 0x7f06f806b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.673+0000 7f06fcf23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f06f81072f0 con 0x7f06f81013b0 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.673+0000 7f06f659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f81013b0 0x7f06f806b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.673+0000 7f06f659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f81013b0 0x7f06f806b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54112/0 (socket says 172.21.15.132:54112) 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.673+0000 7f06f659c700 1 -- 172.21.15.132:0/1989192248 learned_addr learned my addr 172.21.15.132:0/1989192248 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.673+0000 7f06f659c700 1 -- 172.21.15.132:0/1989192248 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f819a820 con 0x7f06f81013b0 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.673+0000 7f06f659c700 1 --2- 172.21.15.132:0/1989192248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f81013b0 0x7f06f806b360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06e8004fb0 tx=0x7f06e8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.674+0000 7f06e77fe700 1 -- 172.21.15.132:0/1989192248 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e8027070 con 0x7f06f81013b0 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.674+0000 7f06e77fe700 1 -- 172.21.15.132:0/1989192248 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e800be10 con 0x7f06f81013b0 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.674+0000 7f06fcf23700 1 -- 172.21.15.132:0/1989192248 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f819a9f0 con 0x7f06f81013b0 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.674+0000 7f06e77fe700 1 -- 172.21.15.132:0/1989192248 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e8022de0 con 0x7f06f81013b0 2024-04-23T18:27:09.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.674+0000 7f06fcf23700 1 -- 172.21.15.132:0/1989192248 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f819ae10 con 0x7f06f81013b0 2024-04-23T18:27:09.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.675+0000 7f06e77fe700 1 -- 172.21.15.132:0/1989192248 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f06e802a070 con 0x7f06f81013b0 2024-04-23T18:27:09.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.675+0000 7f06fcf23700 1 -- 172.21.15.132:0/1989192248 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f8107a70 con 0x7f06f81013b0 2024-04-23T18:27:09.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.675+0000 7f06e77fe700 1 --2- 172.21.15.132:0/1989192248 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f06e0039650 0x7f06e003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:09.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.675+0000 7f06e77fe700 1 -- 172.21.15.132:0/1989192248 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f06e801e030 con 0x7f06f81013b0 2024-04-23T18:27:09.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.675+0000 7f06f5d9b700 1 --2- 172.21.15.132:0/1989192248 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f06e0039650 0x7f06e003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:09.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.675+0000 7f06f5d9b700 1 --2- 172.21.15.132:0/1989192248 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f06e0039650 0x7f06e003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f06ec006fd0 tx=0x7f06ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:09.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.680+0000 7f06e77fe700 1 -- 172.21.15.132:0/1989192248 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06e8013070 con 0x7f06f81013b0 2024-04-23T18:27:09.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.881+0000 7f06fcf23700 1 -- 172.21.15.132:0/1989192248 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f06f802e1b0 con 0x7f06e0039650 2024-04-23T18:27:09.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.885+0000 7f06e77fe700 1 -- 172.21.15.132:0/1989192248 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f06f802e1b0 con 0x7f06e0039650 2024-04-23T18:27:09.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.887+0000 7f06fcf23700 1 -- 172.21.15.132:0/1989192248 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f06e0039650 msgr2=0x7f06e003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:09.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.887+0000 7f06fcf23700 1 --2- 172.21.15.132:0/1989192248 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f06e0039650 0x7f06e003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f06ec006fd0 tx=0x7f06ec006e40 comp rx=0 tx=0).stop 2024-04-23T18:27:09.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.887+0000 7f06fcf23700 1 -- 172.21.15.132:0/1989192248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f81013b0 msgr2=0x7f06f806b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:09.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.887+0000 7f06fcf23700 1 --2- 172.21.15.132:0/1989192248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f81013b0 0x7f06f806b360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06e8004fb0 tx=0x7f06e8009f60 comp rx=0 tx=0).stop 2024-04-23T18:27:09.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.887+0000 7f06fcf23700 1 -- 172.21.15.132:0/1989192248 shutdown_connections 2024-04-23T18:27:09.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.887+0000 7f06fcf23700 1 --2- 172.21.15.132:0/1989192248 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f06e0039650 0x7f06e003bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:09.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.887+0000 7f06fcf23700 1 --2- 172.21.15.132:0/1989192248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06f81013b0 0x7f06f806b360 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:09.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.887+0000 7f06fcf23700 1 -- 172.21.15.132:0/1989192248 >> 172.21.15.132:0/1989192248 conn(0x7f06f80fc950 msgr2=0x7f06f80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:09.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.887+0000 7f06fcf23700 1 -- 172.21.15.132:0/1989192248 shutdown_connections 2024-04-23T18:27:09.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:09.887+0000 7f06fcf23700 1 -- 172.21.15.132:0/1989192248 wait complete. 2024-04-23T18:27:10.665 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi148 2024-04-23T18:27:10.666 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T18:27:10.666 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.conf 2024-04-23T18:27:10.685 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T18:27:10.685 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:27:10.744 INFO:tasks.cephadm:Adding host smithi148 to orchestrator... 2024-04-23T18:27:10.744 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch host add smithi148 2024-04-23T18:27:10.979 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:27:11.101 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:10 smithi132 ceph-mon[29178]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:27:11.101 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:11.101 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:11.101 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:11.101 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:11.102 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:10 smithi132 ceph-mon[29178]: Updating smithi132:/etc/ceph/ceph.conf 2024-04-23T18:27:11.102 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:10 smithi132 ceph-mon[29178]: Updating smithi132:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:11.102 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:11.102 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:11.102 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:12.007 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:11 smithi132 ceph-mon[29178]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:27:12.007 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:11 smithi132 ceph-mon[29178]: Updating smithi132:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:27:12.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.451+0000 7fb49967a700 1 -- 172.21.15.132:0/1990099009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb494103640 msgr2=0x7fb494103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:12.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.451+0000 7fb49967a700 1 --2- 172.21.15.132:0/1990099009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb494103640 0x7fb494103a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb484009910 tx=0x7fb484009c20 comp rx=0 tx=0).stop 2024-04-23T18:27:12.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.451+0000 7fb49967a700 1 -- 172.21.15.132:0/1990099009 shutdown_connections 2024-04-23T18:27:12.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.451+0000 7fb49967a700 1 --2- 172.21.15.132:0/1990099009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb494103640 0x7fb494103a20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:12.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.451+0000 7fb49967a700 1 -- 172.21.15.132:0/1990099009 >> 172.21.15.132:0/1990099009 conn(0x7fb494078010 msgr2=0x7fb4940ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:12.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.451+0000 7fb49967a700 1 -- 172.21.15.132:0/1990099009 shutdown_connections 2024-04-23T18:27:12.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.451+0000 7fb49967a700 1 -- 172.21.15.132:0/1990099009 wait complete. 2024-04-23T18:27:12.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.452+0000 7fb49967a700 1 Processor -- start 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.452+0000 7fb49967a700 1 -- start start 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.452+0000 7fb49967a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb494103640 0x7fb494197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.452+0000 7fb49967a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb49410c970 con 0x7fb494103640 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.453+0000 7fb492ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb494103640 0x7fb494197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.453+0000 7fb492ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb494103640 0x7fb494197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54128/0 (socket says 172.21.15.132:54128) 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.453+0000 7fb492ffd700 1 -- 172.21.15.132:0/680724012 learned_addr learned my addr 172.21.15.132:0/680724012 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.453+0000 7fb492ffd700 1 -- 172.21.15.132:0/680724012 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb494198400 con 0x7fb494103640 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.453+0000 7fb492ffd700 1 --2- 172.21.15.132:0/680724012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb494103640 0x7fb494197ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb484006010 tx=0x7fb484005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.453+0000 7fb47bfff700 1 -- 172.21.15.132:0/680724012 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb484026070 con 0x7fb494103640 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.453+0000 7fb47bfff700 1 -- 172.21.15.132:0/680724012 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb48400bc00 con 0x7fb494103640 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.453+0000 7fb47bfff700 1 -- 172.21.15.132:0/680724012 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb484003910 con 0x7fb494103640 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.453+0000 7fb49967a700 1 -- 172.21.15.132:0/680724012 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb494198690 con 0x7fb494103640 2024-04-23T18:27:12.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.453+0000 7fb49967a700 1 -- 172.21.15.132:0/680724012 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb494198af0 con 0x7fb494103640 2024-04-23T18:27:12.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.454+0000 7fb47bfff700 1 -- 172.21.15.132:0/680724012 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fb484029070 con 0x7fb494103640 2024-04-23T18:27:12.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.454+0000 7fb49967a700 1 -- 172.21.15.132:0/680724012 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb49406a6f0 con 0x7fb494103640 2024-04-23T18:27:12.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.454+0000 7fb47bfff700 1 --2- 172.21.15.132:0/680724012 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb47c0396b0 0x7fb47c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:12.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.454+0000 7fb47bfff700 1 -- 172.21.15.132:0/680724012 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb48401e070 con 0x7fb494103640 2024-04-23T18:27:12.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.454+0000 7fb4927fc700 1 --2- 172.21.15.132:0/680724012 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb47c0396b0 0x7fb47c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:12.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.455+0000 7fb4927fc700 1 --2- 172.21.15.132:0/680724012 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb47c0396b0 0x7fb47c03bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb488006fd0 tx=0x7fb488006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:12.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.459+0000 7fb47bfff700 1 -- 172.21.15.132:0/680724012 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb484013070 con 0x7fb494103640 2024-04-23T18:27:12.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:12.663+0000 7fb49967a700 1 -- 172.21.15.132:0/680724012 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}) -- 0x7fb494199290 con 0x7fb47c0396b0 2024-04-23T18:27:13.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:13 smithi132 ceph-mon[29178]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:27:15.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:15 smithi132 ceph-mon[29178]: Deploying cephadm binary to smithi148 2024-04-23T18:27:15.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:15 smithi132 ceph-mon[29178]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:16.781 INFO:teuthology.orchestra.run.smithi132.stdout:Added host 'smithi148' with addr '172.21.15.148' 2024-04-23T18:27:16.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:16.788+0000 7fb47bfff700 1 -- 172.21.15.132:0/680724012 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb494199290 con 0x7fb47c0396b0 2024-04-23T18:27:16.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:16.791+0000 7fb49967a700 1 -- 172.21.15.132:0/680724012 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb47c0396b0 msgr2=0x7fb47c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:16.791+0000 7fb49967a700 1 --2- 172.21.15.132:0/680724012 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb47c0396b0 0x7fb47c03bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb488006fd0 tx=0x7fb488006e40 comp rx=0 tx=0).stop 2024-04-23T18:27:16.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:16.791+0000 7fb49967a700 1 -- 172.21.15.132:0/680724012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb494103640 msgr2=0x7fb494197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:16.791+0000 7fb49967a700 1 --2- 172.21.15.132:0/680724012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb494103640 0x7fb494197ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb484006010 tx=0x7fb484005780 comp rx=0 tx=0).stop 2024-04-23T18:27:16.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:16.791+0000 7fb49967a700 1 -- 172.21.15.132:0/680724012 shutdown_connections 2024-04-23T18:27:16.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:16.791+0000 7fb49967a700 1 --2- 172.21.15.132:0/680724012 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb47c0396b0 0x7fb47c03bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:16.791+0000 7fb49967a700 1 --2- 172.21.15.132:0/680724012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb494103640 0x7fb494197ec0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:16.791+0000 7fb49967a700 1 -- 172.21.15.132:0/680724012 >> 172.21.15.132:0/680724012 conn(0x7fb494078010 msgr2=0x7fb4941043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:16.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:16.791+0000 7fb49967a700 1 -- 172.21.15.132:0/680724012 shutdown_connections 2024-04-23T18:27:16.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:16.791+0000 7fb49967a700 1 -- 172.21.15.132:0/680724012 wait complete. 2024-04-23T18:27:17.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:17 smithi132 ceph-mon[29178]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:17.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:17 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:17.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:17 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:17.436 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T18:27:17.672 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:27:18.301 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:18 smithi132 ceph-mon[29178]: Added host smithi148 2024-04-23T18:27:18.302 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:18 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:18.302 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:18 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:19.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.105+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2707541067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8108490 msgr2=0x7fa3c8108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:19.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.105+0000 7fa3ccfa5700 1 --2- 172.21.15.132:0/2707541067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8108490 0x7fa3c8108870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009910 tx=0x7fa3b8009c20 comp rx=0 tx=0).stop 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.105+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2707541067 shutdown_connections 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.105+0000 7fa3ccfa5700 1 --2- 172.21.15.132:0/2707541067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8108490 0x7fa3c8108870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.105+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2707541067 >> 172.21.15.132:0/2707541067 conn(0x7fa3c8077a30 msgr2=0x7fa3c8077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.105+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2707541067 shutdown_connections 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.105+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2707541067 wait complete. 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.106+0000 7fa3ccfa5700 1 Processor -- start 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.106+0000 7fa3ccfa5700 1 -- start start 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.106+0000 7fa3ccfa5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8108490 0x7fa3c8197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.106+0000 7fa3ccfa5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c810c880 con 0x7fa3c8108490 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.106+0000 7fa3c659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8108490 0x7fa3c8197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.106+0000 7fa3c659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8108490 0x7fa3c8197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54132/0 (socket says 172.21.15.132:54132) 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.106+0000 7fa3c659c700 1 -- 172.21.15.132:0/2570825462 learned_addr learned my addr 172.21.15.132:0/2570825462 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:19.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.106+0000 7fa3c659c700 1 -- 172.21.15.132:0/2570825462 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c81982e0 con 0x7fa3c8108490 2024-04-23T18:27:19.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.107+0000 7fa3c659c700 1 --2- 172.21.15.132:0/2570825462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8108490 0x7fa3c8197da0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80056d0 tx=0x7fa3b80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:19.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.107+0000 7fa3bffff700 1 -- 172.21.15.132:0/2570825462 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b8026070 con 0x7fa3c8108490 2024-04-23T18:27:19.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.107+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2570825462 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c81985d0 con 0x7fa3c8108490 2024-04-23T18:27:19.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.107+0000 7fa3bffff700 1 -- 172.21.15.132:0/2570825462 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b800bc00 con 0x7fa3c8108490 2024-04-23T18:27:19.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.107+0000 7fa3bffff700 1 -- 172.21.15.132:0/2570825462 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b8003920 con 0x7fa3c8108490 2024-04-23T18:27:19.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.107+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2570825462 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c819c670 con 0x7fa3c8108490 2024-04-23T18:27:19.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.108+0000 7fa3bffff700 1 -- 172.21.15.132:0/2570825462 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fa3b8029070 con 0x7fa3c8108490 2024-04-23T18:27:19.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.108+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2570825462 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c8103d50 con 0x7fa3c8108490 2024-04-23T18:27:19.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.108+0000 7fa3bffff700 1 --2- 172.21.15.132:0/2570825462 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa3b4039650 0x7fa3b403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:19.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.108+0000 7fa3bffff700 1 -- 172.21.15.132:0/2570825462 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa3b801e070 con 0x7fa3c8108490 2024-04-23T18:27:19.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.108+0000 7fa3bdd9b700 1 --2- 172.21.15.132:0/2570825462 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa3b4039650 0x7fa3b403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:19.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.108+0000 7fa3bdd9b700 1 --2- 172.21.15.132:0/2570825462 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa3b4039650 0x7fa3b403bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0006fd0 tx=0x7fa3b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:19.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.113+0000 7fa3bffff700 1 -- 172.21.15.132:0/2570825462 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3b8013070 con 0x7fa3c8108490 2024-04-23T18:27:19.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.314+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2570825462 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3c8199100 con 0x7fa3b4039650 2024-04-23T18:27:19.308 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:19 smithi132 ceph-mon[29178]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:19.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.316+0000 7fa3bffff700 1 -- 172.21.15.132:0/2570825462 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fa3c8199100 con 0x7fa3b4039650 2024-04-23T18:27:19.308 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:27:19.308 INFO:teuthology.orchestra.run.smithi132.stdout:[{"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}] 2024-04-23T18:27:19.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.318+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2570825462 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa3b4039650 msgr2=0x7fa3b403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:19.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.318+0000 7fa3ccfa5700 1 --2- 172.21.15.132:0/2570825462 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa3b4039650 0x7fa3b403bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0006fd0 tx=0x7fa3b0006e40 comp rx=0 tx=0).stop 2024-04-23T18:27:19.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.318+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2570825462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8108490 msgr2=0x7fa3c8197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:19.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.318+0000 7fa3ccfa5700 1 --2- 172.21.15.132:0/2570825462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8108490 0x7fa3c8197da0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80056d0 tx=0x7fa3b80057b0 comp rx=0 tx=0).stop 2024-04-23T18:27:19.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.318+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2570825462 shutdown_connections 2024-04-23T18:27:19.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.318+0000 7fa3ccfa5700 1 --2- 172.21.15.132:0/2570825462 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa3b4039650 0x7fa3b403bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:19.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.318+0000 7fa3ccfa5700 1 --2- 172.21.15.132:0/2570825462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8108490 0x7fa3c8197da0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:19.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.318+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2570825462 >> 172.21.15.132:0/2570825462 conn(0x7fa3c8077a30 msgr2=0x7fa3c80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:19.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.318+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2570825462 shutdown_connections 2024-04-23T18:27:19.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:19.318+0000 7fa3ccfa5700 1 -- 172.21.15.132:0/2570825462 wait complete. 2024-04-23T18:27:19.970 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2024-04-23T18:27:19.971 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-23T18:27:19.971 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2024-04-23T18:27:19.990 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-23T18:27:19.990 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:27:20.048 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2024-04-23T18:27:20.048 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch host add smithi157 2024-04-23T18:27:20.280 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:27:20.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:20 smithi132 ceph-mon[29178]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:27:20.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:20.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:20 smithi132 ceph-mon[29178]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:21.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.726+0000 7f5d08883700 1 -- 172.21.15.132:0/1281893389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d041084c0 msgr2=0x7f5d041088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.726+0000 7f5d08883700 1 --2- 172.21.15.132:0/1281893389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d041084c0 0x7f5d041088a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009910 tx=0x7f5cf8009c20 comp rx=0 tx=0).stop 2024-04-23T18:27:21.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.727+0000 7f5d08883700 1 -- 172.21.15.132:0/1281893389 shutdown_connections 2024-04-23T18:27:21.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.727+0000 7f5d08883700 1 --2- 172.21.15.132:0/1281893389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d041084c0 0x7f5d041088a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.727+0000 7f5d08883700 1 -- 172.21.15.132:0/1281893389 >> 172.21.15.132:0/1281893389 conn(0x7f5d040fdf60 msgr2=0x7f5d04100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.727+0000 7f5d08883700 1 -- 172.21.15.132:0/1281893389 shutdown_connections 2024-04-23T18:27:21.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.727+0000 7f5d08883700 1 -- 172.21.15.132:0/1281893389 wait complete. 2024-04-23T18:27:21.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.727+0000 7f5d08883700 1 Processor -- start 2024-04-23T18:27:21.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.727+0000 7f5d08883700 1 -- start start 2024-04-23T18:27:21.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.728+0000 7f5d08883700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d041084c0 0x7f5d04197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.728+0000 7f5d08883700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0410c920 con 0x7f5d041084c0 2024-04-23T18:27:21.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.728+0000 7f5d0259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d041084c0 0x7f5d04197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.728+0000 7f5d0259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d041084c0 0x7f5d04197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43782/0 (socket says 172.21.15.132:43782) 2024-04-23T18:27:21.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.728+0000 7f5d0259c700 1 -- 172.21.15.132:0/1511436406 learned_addr learned my addr 172.21.15.132:0/1511436406 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:21.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.728+0000 7f5d0259c700 1 -- 172.21.15.132:0/1511436406 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d041983b0 con 0x7f5d041084c0 2024-04-23T18:27:21.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.728+0000 7f5d0259c700 1 --2- 172.21.15.132:0/1511436406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d041084c0 0x7f5d04197e70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80056d0 tx=0x7f5cf80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.728+0000 7f5cf37fe700 1 -- 172.21.15.132:0/1511436406 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cf8026070 con 0x7f5d041084c0 2024-04-23T18:27:21.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.728+0000 7f5cf37fe700 1 -- 172.21.15.132:0/1511436406 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf800bba0 con 0x7f5d041084c0 2024-04-23T18:27:21.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.728+0000 7f5d08883700 1 -- 172.21.15.132:0/1511436406 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d041986a0 con 0x7f5d041084c0 2024-04-23T18:27:21.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.728+0000 7f5cf37fe700 1 -- 172.21.15.132:0/1511436406 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cf8003920 con 0x7f5d041084c0 2024-04-23T18:27:21.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.728+0000 7f5d08883700 1 -- 172.21.15.132:0/1511436406 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0419c740 con 0x7f5d041084c0 2024-04-23T18:27:21.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.729+0000 7f5cf37fe700 1 -- 172.21.15.132:0/1511436406 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f5cf8029070 con 0x7f5d041084c0 2024-04-23T18:27:21.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.729+0000 7f5d08883700 1 -- 172.21.15.132:0/1511436406 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d04103b70 con 0x7f5d041084c0 2024-04-23T18:27:21.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.729+0000 7f5cf37fe700 1 --2- 172.21.15.132:0/1511436406 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5cec039650 0x7f5cec03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.729+0000 7f5cf37fe700 1 -- 172.21.15.132:0/1511436406 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5cf801e070 con 0x7f5d041084c0 2024-04-23T18:27:21.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.729+0000 7f5d01d9b700 1 --2- 172.21.15.132:0/1511436406 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5cec039650 0x7f5cec03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.730+0000 7f5d01d9b700 1 --2- 172.21.15.132:0/1511436406 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5cec039650 0x7f5cec03bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4006fd0 tx=0x7f5cf4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.734+0000 7f5cf37fe700 1 -- 172.21.15.132:0/1511436406 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5cf8013070 con 0x7f5d041084c0 2024-04-23T18:27:21.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:21.937+0000 7f5d08883700 1 -- 172.21.15.132:0/1511436406 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}) -- 0x7f5d041991d0 con 0x7f5cec039650 2024-04-23T18:27:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:23 smithi132 ceph-mon[29178]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:27:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:23 smithi132 ceph-mon[29178]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:27:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:24.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:24 smithi132 ceph-mon[29178]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-23T18:27:24.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:24 smithi132 ceph-mon[29178]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:24.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:24 smithi132 ceph-mon[29178]: Deploying cephadm binary to smithi157 2024-04-23T18:27:24.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:24 smithi132 ceph-mon[29178]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:27:24.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:24 smithi132 ceph-mon[29178]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:27:25.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:25 smithi132 ceph-mon[29178]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:26.143 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:26.148+0000 7f5cf37fe700 1 -- 172.21.15.132:0/1511436406 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5d041991d0 con 0x7f5cec039650 2024-04-23T18:27:26.143 INFO:teuthology.orchestra.run.smithi132.stdout:Added host 'smithi157' with addr '172.21.15.157' 2024-04-23T18:27:26.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:26.152+0000 7f5d08883700 1 -- 172.21.15.132:0/1511436406 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5cec039650 msgr2=0x7f5cec03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:26.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:26.152+0000 7f5d08883700 1 --2- 172.21.15.132:0/1511436406 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5cec039650 0x7f5cec03bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4006fd0 tx=0x7f5cf4006e40 comp rx=0 tx=0).stop 2024-04-23T18:27:26.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:26.152+0000 7f5d08883700 1 -- 172.21.15.132:0/1511436406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d041084c0 msgr2=0x7f5d04197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:26.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:26.152+0000 7f5d08883700 1 --2- 172.21.15.132:0/1511436406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d041084c0 0x7f5d04197e70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80056d0 tx=0x7f5cf80057b0 comp rx=0 tx=0).stop 2024-04-23T18:27:26.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:26.152+0000 7f5d08883700 1 -- 172.21.15.132:0/1511436406 shutdown_connections 2024-04-23T18:27:26.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:26.152+0000 7f5d08883700 1 --2- 172.21.15.132:0/1511436406 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5cec039650 0x7f5cec03bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:26.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:26.152+0000 7f5d08883700 1 --2- 172.21.15.132:0/1511436406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d041084c0 0x7f5d04197e70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:26.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:26.152+0000 7f5d08883700 1 -- 172.21.15.132:0/1511436406 >> 172.21.15.132:0/1511436406 conn(0x7f5d040fdf60 msgr2=0x7f5d040fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:26.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:26.152+0000 7f5d08883700 1 -- 172.21.15.132:0/1511436406 shutdown_connections 2024-04-23T18:27:26.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:26.152+0000 7f5d08883700 1 -- 172.21.15.132:0/1511436406 wait complete. 2024-04-23T18:27:26.762 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:26.762 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:26.762 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:26.762 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:26 smithi132 ceph-mon[29178]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:26.762 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:26.762 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:26 smithi132 ceph-mon[29178]: Added host smithi157 2024-04-23T18:27:26.762 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:26.789 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T18:27:27.023 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:27:27.793 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:27 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:27.793 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:27 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:28.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.538+0000 7efed29c3700 1 -- 172.21.15.132:0/1179161354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efecc108430 msgr2=0x7efecc108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:28.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.538+0000 7efed29c3700 1 --2- 172.21.15.132:0/1179161354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efecc108430 0x7efecc108810 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efec0009910 tx=0x7efec0009c20 comp rx=0 tx=0).stop 2024-04-23T18:27:28.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.538+0000 7efed29c3700 1 -- 172.21.15.132:0/1179161354 shutdown_connections 2024-04-23T18:27:28.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.538+0000 7efed29c3700 1 --2- 172.21.15.132:0/1179161354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efecc108430 0x7efecc108810 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:28.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.538+0000 7efed29c3700 1 -- 172.21.15.132:0/1179161354 >> 172.21.15.132:0/1179161354 conn(0x7efecc078940 msgr2=0x7efecc078d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:28.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.538+0000 7efed29c3700 1 -- 172.21.15.132:0/1179161354 shutdown_connections 2024-04-23T18:27:28.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.538+0000 7efed29c3700 1 -- 172.21.15.132:0/1179161354 wait complete. 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.538+0000 7efed29c3700 1 Processor -- start 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.538+0000 7efed29c3700 1 -- start start 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.539+0000 7efed29c3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efecc108430 0x7efecc197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.539+0000 7efed29c3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7efecc10c920 con 0x7efecc108430 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.539+0000 7efecbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efecc108430 0x7efecc197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.539+0000 7efecbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efecc108430 0x7efecc197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43808/0 (socket says 172.21.15.132:43808) 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.539+0000 7efecbfff700 1 -- 172.21.15.132:0/2117918048 learned_addr learned my addr 172.21.15.132:0/2117918048 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.539+0000 7efecbfff700 1 -- 172.21.15.132:0/2117918048 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc198320 con 0x7efecc108430 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.539+0000 7efecbfff700 1 --2- 172.21.15.132:0/2117918048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efecc108430 0x7efecc197de0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efec00056d0 tx=0x7efec00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.539+0000 7efec97fa700 1 -- 172.21.15.132:0/2117918048 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efec0026070 con 0x7efecc108430 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.539+0000 7efec97fa700 1 -- 172.21.15.132:0/2117918048 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec000bba0 con 0x7efecc108430 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.539+0000 7efec97fa700 1 -- 172.21.15.132:0/2117918048 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efec0003920 con 0x7efecc108430 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.539+0000 7efed29c3700 1 -- 172.21.15.132:0/2117918048 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efecc198610 con 0x7efecc108430 2024-04-23T18:27:28.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.540+0000 7efed29c3700 1 -- 172.21.15.132:0/2117918048 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efecc19c6b0 con 0x7efecc108430 2024-04-23T18:27:28.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.540+0000 7efec97fa700 1 -- 172.21.15.132:0/2117918048 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7efec0029070 con 0x7efecc108430 2024-04-23T18:27:28.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.540+0000 7efed29c3700 1 -- 172.21.15.132:0/2117918048 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efecc06a5d0 con 0x7efecc108430 2024-04-23T18:27:28.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.540+0000 7efec97fa700 1 --2- 172.21.15.132:0/2117918048 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7efeb4039650 0x7efeb403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:28.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.541+0000 7efec97fa700 1 -- 172.21.15.132:0/2117918048 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efec001e070 con 0x7efecc108430 2024-04-23T18:27:28.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.541+0000 7efecb7fe700 1 --2- 172.21.15.132:0/2117918048 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7efeb4039650 0x7efeb403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:28.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.541+0000 7efecb7fe700 1 --2- 172.21.15.132:0/2117918048 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7efeb4039650 0x7efeb403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efebc006fd0 tx=0x7efebc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:28.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.550+0000 7efec97fa700 1 -- 172.21.15.132:0/2117918048 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efec0013070 con 0x7efecc108430 2024-04-23T18:27:28.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.759+0000 7efed29c3700 1 -- 172.21.15.132:0/2117918048 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efecc1991b0 con 0x7efeb4039650 2024-04-23T18:27:28.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.760+0000 7efec97fa700 1 -- 172.21.15.132:0/2117918048 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7efecc1991b0 con 0x7efeb4039650 2024-04-23T18:27:28.753 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:27:28.753 INFO:teuthology.orchestra.run.smithi132.stdout:[{"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2024-04-23T18:27:28.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.762+0000 7efed29c3700 1 -- 172.21.15.132:0/2117918048 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7efeb4039650 msgr2=0x7efeb403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:28.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.762+0000 7efed29c3700 1 --2- 172.21.15.132:0/2117918048 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7efeb4039650 0x7efeb403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efebc006fd0 tx=0x7efebc006e40 comp rx=0 tx=0).stop 2024-04-23T18:27:28.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.762+0000 7efed29c3700 1 -- 172.21.15.132:0/2117918048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efecc108430 msgr2=0x7efecc197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:28.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.762+0000 7efed29c3700 1 --2- 172.21.15.132:0/2117918048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efecc108430 0x7efecc197de0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efec00056d0 tx=0x7efec00057b0 comp rx=0 tx=0).stop 2024-04-23T18:27:28.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.762+0000 7efed29c3700 1 -- 172.21.15.132:0/2117918048 shutdown_connections 2024-04-23T18:27:28.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.762+0000 7efed29c3700 1 --2- 172.21.15.132:0/2117918048 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7efeb4039650 0x7efeb403bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:28.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.762+0000 7efed29c3700 1 --2- 172.21.15.132:0/2117918048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efecc108430 0x7efecc197de0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:28.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.762+0000 7efed29c3700 1 -- 172.21.15.132:0/2117918048 >> 172.21.15.132:0/2117918048 conn(0x7efecc078940 msgr2=0x7efecc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:28.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.762+0000 7efed29c3700 1 -- 172.21.15.132:0/2117918048 shutdown_connections 2024-04-23T18:27:28.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:28.762+0000 7efed29c3700 1 -- 172.21.15.132:0/2117918048 wait complete. 2024-04-23T18:27:29.048 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:28 smithi132 ceph-mon[29178]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:29.318 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-23T18:27:29.319 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-23T18:27:29.551 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:27:30.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:30 smithi132 ceph-mon[29178]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:27:30.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:30 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:30.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.958+0000 7f63191a6700 1 -- 172.21.15.132:0/3870077784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6314106340 msgr2=0x7f6314106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:30.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.958+0000 7f63191a6700 1 --2- 172.21.15.132:0/3870077784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6314106340 0x7f6314106720 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009910 tx=0x7f62fc009c20 comp rx=0 tx=0).stop 2024-04-23T18:27:30.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.958+0000 7f63191a6700 1 -- 172.21.15.132:0/3870077784 shutdown_connections 2024-04-23T18:27:30.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.958+0000 7f63191a6700 1 --2- 172.21.15.132:0/3870077784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6314106340 0x7f6314106720 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:30.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.958+0000 7f63191a6700 1 -- 172.21.15.132:0/3870077784 >> 172.21.15.132:0/3870077784 conn(0x7f6314077b30 msgr2=0x7f6314077f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:30.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.958+0000 7f63191a6700 1 -- 172.21.15.132:0/3870077784 shutdown_connections 2024-04-23T18:27:30.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.958+0000 7f63191a6700 1 -- 172.21.15.132:0/3870077784 wait complete. 2024-04-23T18:27:30.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.959+0000 7f63191a6700 1 Processor -- start 2024-04-23T18:27:30.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.959+0000 7f63191a6700 1 -- start start 2024-04-23T18:27:30.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.960+0000 7f63191a6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6314106340 0x7f6314198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:30.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.960+0000 7f63191a6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f631410b970 con 0x7f6314106340 2024-04-23T18:27:30.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.960+0000 7f6312d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6314106340 0x7f6314198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:30.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.960+0000 7f6312d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6314106340 0x7f6314198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37430/0 (socket says 172.21.15.132:37430) 2024-04-23T18:27:30.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.960+0000 7f6312d9d700 1 -- 172.21.15.132:0/1786483791 learned_addr learned my addr 172.21.15.132:0/1786483791 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:30.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.960+0000 7f6312d9d700 1 -- 172.21.15.132:0/1786483791 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6314199260 con 0x7f6314106340 2024-04-23T18:27:30.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.960+0000 7f6312d9d700 1 --2- 172.21.15.132:0/1786483791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6314106340 0x7f6314198d20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0056d0 tx=0x7f62fc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:30.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.960+0000 7f630bfff700 1 -- 172.21.15.132:0/1786483791 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62fc026070 con 0x7f6314106340 2024-04-23T18:27:30.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.960+0000 7f630bfff700 1 -- 172.21.15.132:0/1786483791 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62fc00bc00 con 0x7f6314106340 2024-04-23T18:27:30.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.960+0000 7f63191a6700 1 -- 172.21.15.132:0/1786483791 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63141994f0 con 0x7f6314106340 2024-04-23T18:27:30.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.960+0000 7f630bfff700 1 -- 172.21.15.132:0/1786483791 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62fc003920 con 0x7f6314106340 2024-04-23T18:27:30.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.960+0000 7f63191a6700 1 -- 172.21.15.132:0/1786483791 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6314199950 con 0x7f6314106340 2024-04-23T18:27:30.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.961+0000 7f630bfff700 1 -- 172.21.15.132:0/1786483791 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f62fc029070 con 0x7f6314106340 2024-04-23T18:27:30.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.961+0000 7f63191a6700 1 -- 172.21.15.132:0/1786483791 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6314102b00 con 0x7f6314106340 2024-04-23T18:27:30.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.961+0000 7f630bfff700 1 --2- 172.21.15.132:0/1786483791 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6300039650 0x7f630003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:30.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.962+0000 7f630bfff700 1 -- 172.21.15.132:0/1786483791 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f62fc01e070 con 0x7f6314106340 2024-04-23T18:27:30.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.962+0000 7f631259c700 1 --2- 172.21.15.132:0/1786483791 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6300039650 0x7f630003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:30.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.962+0000 7f631259c700 1 --2- 172.21.15.132:0/1786483791 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6300039650 0x7f630003bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6304006fd0 tx=0x7f6304006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:30.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:30.967+0000 7f630bfff700 1 -- 172.21.15.132:0/1786483791 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62fc013070 con 0x7f6314106340 2024-04-23T18:27:31.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:31 smithi132 ceph-mon[29178]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:31.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:31.171+0000 7f63191a6700 1 -- 172.21.15.132:0/1786483791 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f63140640a0 con 0x7f6314106340 2024-04-23T18:27:32.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:32.038+0000 7f630bfff700 1 -- 172.21.15.132:0/1786483791 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f62fc030090 con 0x7f6314106340 2024-04-23T18:27:32.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:32.041+0000 7f63191a6700 1 -- 172.21.15.132:0/1786483791 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6300039650 msgr2=0x7f630003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:32.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:32.041+0000 7f63191a6700 1 --2- 172.21.15.132:0/1786483791 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6300039650 0x7f630003bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6304006fd0 tx=0x7f6304006e40 comp rx=0 tx=0).stop 2024-04-23T18:27:32.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:32.042+0000 7f63191a6700 1 -- 172.21.15.132:0/1786483791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6314106340 msgr2=0x7f6314198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:32.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:32.042+0000 7f63191a6700 1 --2- 172.21.15.132:0/1786483791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6314106340 0x7f6314198d20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0056d0 tx=0x7f62fc0057b0 comp rx=0 tx=0).stop 2024-04-23T18:27:32.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:32.042+0000 7f63191a6700 1 -- 172.21.15.132:0/1786483791 shutdown_connections 2024-04-23T18:27:32.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:32.042+0000 7f63191a6700 1 --2- 172.21.15.132:0/1786483791 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6300039650 0x7f630003bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:32.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:32.042+0000 7f63191a6700 1 --2- 172.21.15.132:0/1786483791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6314106340 0x7f6314198d20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:32.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:32.042+0000 7f63191a6700 1 -- 172.21.15.132:0/1786483791 >> 172.21.15.132:0/1786483791 conn(0x7f6314077b30 msgr2=0x7f6314108410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:32.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:32.042+0000 7f63191a6700 1 -- 172.21.15.132:0/1786483791 shutdown_connections 2024-04-23T18:27:32.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:32.042+0000 7f63191a6700 1 -- 172.21.15.132:0/1786483791 wait complete. 2024-04-23T18:27:32.036 INFO:teuthology.orchestra.run.smithi132.stderr:adjusted tunables profile to default 2024-04-23T18:27:32.345 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:32 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1786483791' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T18:27:32.701 INFO:tasks.cephadm:Adding mon.a on smithi132 2024-04-23T18:27:32.702 INFO:tasks.cephadm:Adding mon.b on smithi148 2024-04-23T18:27:32.702 INFO:tasks.cephadm:Adding mon.c on smithi157 2024-04-23T18:27:32.702 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi132:172.21.15.132=a;smithi148:172.21.15.148=b;smithi157:172.21.15.157=c' 2024-04-23T18:27:32.936 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:33 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1786483791' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T18:27:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:33 smithi132 ceph-mon[29178]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:27:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:33 smithi132 ceph-mon[29178]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:33 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:33 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:33 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:33 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:33.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:33 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:27:33.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:33 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:33.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:33 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:34.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.473+0000 7f9111441700 1 -- 172.21.15.157:0/1481192232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f910c1084d0 msgr2=0x7f910c1088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:34.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.473+0000 7f9111441700 1 --2- 172.21.15.157:0/1481192232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f910c1084d0 0x7f910c1088b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00b0a0 tx=0x7f90fc00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:27:34.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.474+0000 7f9111441700 1 -- 172.21.15.157:0/1481192232 shutdown_connections 2024-04-23T18:27:34.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.474+0000 7f9111441700 1 --2- 172.21.15.157:0/1481192232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f910c1084d0 0x7f910c1088b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:34.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.474+0000 7f9111441700 1 -- 172.21.15.157:0/1481192232 >> 172.21.15.157:0/1481192232 conn(0x7f910c0fdf30 msgr2=0x7f910c100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:34.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.474+0000 7f9111441700 1 -- 172.21.15.157:0/1481192232 shutdown_connections 2024-04-23T18:27:34.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.474+0000 7f9111441700 1 -- 172.21.15.157:0/1481192232 wait complete. 2024-04-23T18:27:34.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.474+0000 7f9111441700 1 Processor -- start 2024-04-23T18:27:34.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.474+0000 7f9111441700 1 -- start start 2024-04-23T18:27:34.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.474+0000 7f9111441700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f910c1084d0 0x7f910c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:34.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.474+0000 7f9111441700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f910c10c930 con 0x7f910c1084d0 2024-04-23T18:27:34.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.475+0000 7f910affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f910c1084d0 0x7f910c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:34.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.475+0000 7f910affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f910c1084d0 0x7f910c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.157:44942/0 (socket says 172.21.15.157:44942) 2024-04-23T18:27:34.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.475+0000 7f910affd700 1 -- 172.21.15.157:0/1977194896 learned_addr learned my addr 172.21.15.157:0/1977194896 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-23T18:27:34.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.475+0000 7f910affd700 1 -- 172.21.15.157:0/1977194896 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c076c30 con 0x7f910c1084d0 2024-04-23T18:27:34.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.475+0000 7f910affd700 1 --2- 172.21.15.157:0/1977194896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f910c1084d0 0x7f910c074d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f90fc006130 tx=0x7f90fc0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:34.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.475+0000 7f9103fff700 1 -- 172.21.15.157:0/1977194896 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc021020 con 0x7f910c1084d0 2024-04-23T18:27:34.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.475+0000 7f9111441700 1 -- 172.21.15.157:0/1977194896 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910c0752e0 con 0x7f910c1084d0 2024-04-23T18:27:34.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.475+0000 7f9103fff700 1 -- 172.21.15.157:0/1977194896 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90fc0079b0 con 0x7f910c1084d0 2024-04-23T18:27:34.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.475+0000 7f9103fff700 1 -- 172.21.15.157:0/1977194896 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc017040 con 0x7f910c1084d0 2024-04-23T18:27:34.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.476+0000 7f9111441700 1 -- 172.21.15.157:0/1977194896 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910c0757a0 con 0x7f910c1084d0 2024-04-23T18:27:34.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.476+0000 7f9103fff700 1 -- 172.21.15.157:0/1977194896 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f90fc024020 con 0x7f910c1084d0 2024-04-23T18:27:34.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.476+0000 7f9111441700 1 -- 172.21.15.157:0/1977194896 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910c103b80 con 0x7f910c1084d0 2024-04-23T18:27:34.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.477+0000 7f9103fff700 1 --2- 172.21.15.157:0/1977194896 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f90f8039650 0x7f90f803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:34.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.477+0000 7f9103fff700 1 -- 172.21.15.157:0/1977194896 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f90fc00f050 con 0x7f910c1084d0 2024-04-23T18:27:34.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.477+0000 7f910a7fc700 1 --2- 172.21.15.157:0/1977194896 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f90f8039650 0x7f90f803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:34.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.477+0000 7f910a7fc700 1 --2- 172.21.15.157:0/1977194896 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f90f8039650 0x7f90f803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f90f4006fd0 tx=0x7f90f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:34.474 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.482+0000 7f9103fff700 1 -- 172.21.15.157:0/1977194896 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90fc02b020 con 0x7f910c1084d0 2024-04-23T18:27:34.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:34 smithi132 ceph-mon[29178]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-23T18:27:34.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:34 smithi132 ceph-mon[29178]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:34.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:34 smithi132 ceph-mon[29178]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:27:34.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:34 smithi132 ceph-mon[29178]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:27:34.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:34 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:34.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:34 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:34.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:34 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:34.684 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.692+0000 7f9111441700 1 -- 172.21.15.157:0/1977194896 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi132:172.21.15.132=a;smithi148:172.21.15.148=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}) -- 0x7f910c0762e0 con 0x7f90f8039650 2024-04-23T18:27:34.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.699+0000 7f9103fff700 1 -- 172.21.15.157:0/1977194896 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f910c0762e0 con 0x7f90f8039650 2024-04-23T18:27:34.691 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mon update... 2024-04-23T18:27:34.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.701+0000 7f9111441700 1 -- 172.21.15.157:0/1977194896 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f90f8039650 msgr2=0x7f90f803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:34.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.701+0000 7f9111441700 1 --2- 172.21.15.157:0/1977194896 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f90f8039650 0x7f90f803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f90f4006fd0 tx=0x7f90f4006e40 comp rx=0 tx=0).stop 2024-04-23T18:27:34.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.701+0000 7f9111441700 1 -- 172.21.15.157:0/1977194896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f910c1084d0 msgr2=0x7f910c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:34.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.701+0000 7f9111441700 1 --2- 172.21.15.157:0/1977194896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f910c1084d0 0x7f910c074d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f90fc006130 tx=0x7f90fc0097d0 comp rx=0 tx=0).stop 2024-04-23T18:27:34.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.701+0000 7f9111441700 1 -- 172.21.15.157:0/1977194896 shutdown_connections 2024-04-23T18:27:34.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.701+0000 7f9111441700 1 --2- 172.21.15.157:0/1977194896 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f90f8039650 0x7f90f803bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:34.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.701+0000 7f9111441700 1 --2- 172.21.15.157:0/1977194896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f910c1084d0 0x7f910c074d40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:34.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.701+0000 7f9111441700 1 -- 172.21.15.157:0/1977194896 >> 172.21.15.157:0/1977194896 conn(0x7f910c0fdf30 msgr2=0x7f910c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:34.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.701+0000 7f9111441700 1 -- 172.21.15.157:0/1977194896 shutdown_connections 2024-04-23T18:27:34.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:34.701+0000 7f9111441700 1 -- 172.21.15.157:0/1977194896 wait complete. 2024-04-23T18:27:35.329 DEBUG:teuthology.orchestra.run.smithi148:mon.b> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.b.service 2024-04-23T18:27:35.332 DEBUG:teuthology.orchestra.run.smithi157:mon.c> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.c.service 2024-04-23T18:27:35.334 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T18:27:35.334 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T18:27:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:35 smithi132 ceph-mon[29178]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:35.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:27:35.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:36.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:36 smithi132 ceph-mon[29178]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi132:172.21.15.132=a;smithi148:172.21.15.148=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:27:36.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:36 smithi132 ceph-mon[29178]: Saving service mon spec with placement smithi132:172.21.15.132=a;smithi148:172.21.15.148=b;smithi157:172.21.15.157=c;count:3 2024-04-23T18:27:36.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:36 smithi132 ceph-mon[29178]: Deploying daemon mon.c on smithi157 2024-04-23T18:27:37.399 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:27:37.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:37 smithi132 ceph-mon[29178]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:38.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:38 smithi157 systemd[1]: Starting Ceph mon.c for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:27:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.176+0000 7fcb1ae81700 1 -- 172.21.15.157:0/236216089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb141074d0 msgr2=0x7fcb141078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.176+0000 7fcb1ae81700 1 --2- 172.21.15.157:0/236216089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb141074d0 0x7fcb141078b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00b0a0 tx=0x7fcb0c00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:27:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.176+0000 7fcb1ae81700 1 -- 172.21.15.157:0/236216089 shutdown_connections 2024-04-23T18:27:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.176+0000 7fcb1ae81700 1 --2- 172.21.15.157:0/236216089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb141074d0 0x7fcb141078b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.176+0000 7fcb1ae81700 1 -- 172.21.15.157:0/236216089 >> 172.21.15.157:0/236216089 conn(0x7fcb140fcf50 msgr2=0x7fcb140ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.177+0000 7fcb1ae81700 1 -- 172.21.15.157:0/236216089 shutdown_connections 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.177+0000 7fcb1ae81700 1 -- 172.21.15.157:0/236216089 wait complete. 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.177+0000 7fcb1ae81700 1 Processor -- start 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.177+0000 7fcb1ae81700 1 -- start start 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.177+0000 7fcb1ae81700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb141074d0 0x7fcb140fd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.177+0000 7fcb1ae81700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1410b930 con 0x7fcb141074d0 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.178+0000 7fcb18c1d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb141074d0 0x7fcb140fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.178+0000 7fcb18c1d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb141074d0 0x7fcb140fd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.157:48574/0 (socket says 172.21.15.157:48574) 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.178+0000 7fcb18c1d700 1 -- 172.21.15.157:0/2369324343 learned_addr learned my addr 172.21.15.157:0/2369324343 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.178+0000 7fcb18c1d700 1 -- 172.21.15.157:0/2369324343 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb140ff2f0 con 0x7fcb141074d0 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.178+0000 7fcb18c1d700 1 --2- 172.21.15.157:0/2369324343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb141074d0 0x7fcb140fd400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c006130 tx=0x7fcb0c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.178+0000 7fcb11ffb700 1 -- 172.21.15.157:0/2369324343 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb0c021020 con 0x7fcb141074d0 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.178+0000 7fcb1ae81700 1 -- 172.21.15.157:0/2369324343 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb140fd9a0 con 0x7fcb141074d0 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.178+0000 7fcb11ffb700 1 -- 172.21.15.157:0/2369324343 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb0c0079b0 con 0x7fcb141074d0 2024-04-23T18:27:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.178+0000 7fcb11ffb700 1 -- 172.21.15.157:0/2369324343 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb0c017040 con 0x7fcb141074d0 2024-04-23T18:27:39.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.178+0000 7fcb1ae81700 1 -- 172.21.15.157:0/2369324343 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb140fde60 con 0x7fcb141074d0 2024-04-23T18:27:39.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.179+0000 7fcb1ae81700 1 -- 172.21.15.157:0/2369324343 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb14102b80 con 0x7fcb141074d0 2024-04-23T18:27:39.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.179+0000 7fcb11ffb700 1 -- 172.21.15.157:0/2369324343 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fcb0c024020 con 0x7fcb141074d0 2024-04-23T18:27:39.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.180+0000 7fcb11ffb700 1 --2- 172.21.15.157:0/2369324343 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fcb00039650 0x7fcb0003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:39.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.180+0000 7fcb11ffb700 1 -- 172.21.15.157:0/2369324343 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcb0c00f050 con 0x7fcb141074d0 2024-04-23T18:27:39.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.180+0000 7fcb13fff700 1 --2- 172.21.15.157:0/2369324343 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fcb00039650 0x7fcb0003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:39.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.180+0000 7fcb13fff700 1 --2- 172.21.15.157:0/2369324343 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fcb00039650 0x7fcb0003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb04006fd0 tx=0x7fcb04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:39.177 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.185+0000 7fcb11ffb700 1 -- 172.21.15.157:0/2369324343 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb0c02b020 con 0x7fcb141074d0 2024-04-23T18:27:39.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.492+0000 7fcb1ae81700 1 -- 172.21.15.157:0/2369324343 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcb140fe930 con 0x7fcb141074d0 2024-04-23T18:27:39.485 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.493+0000 7fcb11ffb700 1 -- 172.21.15.157:0/2369324343 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fcb0c003f80 con 0x7fcb141074d0 2024-04-23T18:27:39.486 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:27:39.486 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","modified":"2024-04-23T18:25:57.540011Z","created":"2024-04-23T18:25:57.540011Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:3300","nonce":0},{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T18:27:39.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.495+0000 7fcb1ae81700 1 -- 172.21.15.157:0/2369324343 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fcb00039650 msgr2=0x7fcb0003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:39.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.495+0000 7fcb1ae81700 1 --2- 172.21.15.157:0/2369324343 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fcb00039650 0x7fcb0003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb04006fd0 tx=0x7fcb04006e40 comp rx=0 tx=0).stop 2024-04-23T18:27:39.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.495+0000 7fcb1ae81700 1 -- 172.21.15.157:0/2369324343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb141074d0 msgr2=0x7fcb140fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:39.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.495+0000 7fcb1ae81700 1 --2- 172.21.15.157:0/2369324343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb141074d0 0x7fcb140fd400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c006130 tx=0x7fcb0c0097d0 comp rx=0 tx=0).stop 2024-04-23T18:27:39.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.495+0000 7fcb1ae81700 1 -- 172.21.15.157:0/2369324343 shutdown_connections 2024-04-23T18:27:39.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.495+0000 7fcb1ae81700 1 --2- 172.21.15.157:0/2369324343 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fcb00039650 0x7fcb0003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:39.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.495+0000 7fcb1ae81700 1 --2- 172.21.15.157:0/2369324343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb141074d0 0x7fcb140fd400 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:39.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.495+0000 7fcb1ae81700 1 -- 172.21.15.157:0/2369324343 >> 172.21.15.157:0/2369324343 conn(0x7fcb140fcf50 msgr2=0x7fcb1406ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:39.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.495+0000 7fcb1ae81700 1 -- 172.21.15.157:0/2369324343 shutdown_connections 2024-04-23T18:27:39.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:39.495+0000 7fcb1ae81700 1 -- 172.21.15.157:0/2369324343 wait complete. 2024-04-23T18:27:39.489 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-23T18:27:39.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:39 smithi132 ceph-mon[29178]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:39.615 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 podman[32549]: 2024-04-23 18:27:39.275943812 +0000 UTC m=+0.022215214 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:27:39.616 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 podman[32549]: 2024-04-23 18:27:39.402312321 +0000 UTC m=+0.148583704 container create ff4c6dfe43a861932405403ca679b9ce7acadf0fe77d2be10a7e5d07eb678b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-23T18:27:39.616 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 podman[32549]: 2024-04-23 18:27:39.596390577 +0000 UTC m=+0.342661961 container init ff4c6dfe43a861932405403ca679b9ce7acadf0fe77d2be10a7e5d07eb678b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-23T18:27:39.616 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 podman[32549]: 2024-04-23 18:27:39.599938531 +0000 UTC m=+0.346209917 container start ff4c6dfe43a861932405403ca679b9ce7acadf0fe77d2be10a7e5d07eb678b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:27:40.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T18:27:40.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-23T18:27:40.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: pidfile_write: ignore empty --pid-file 2024-04-23T18:27:40.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: load: jerasure load: lrc 2024-04-23T18:27:40.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: RocksDB version: 7.9.2 2024-04-23T18:27:40.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Git sha 0 2024-04-23T18:27:40.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-23T18:27:40.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: DB SUMMARY 2024-04-23T18:27:40.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: DB Session ID: V6VP358YE6ESVJADT2P3 2024-04-23T18:27:40.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: CURRENT file: CURRENT 2024-04-23T18:27:40.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.error_if_exists: 0 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.create_if_missing: 0 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.paranoid_checks: 1 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.env: 0x55d7adaf3cc0 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.info_log: 0x55d7ae905be0 2024-04-23T18:27:40.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.statistics: (nil) 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.use_fsync: 0 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_log_file_size: 0 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.allow_fallocate: 1 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.use_direct_reads: 0 2024-04-23T18:27:40.009 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.db_log_dir: 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.wal_dir: 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.write_buffer_manager: 0x55d7ae8990e0 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T18:27:40.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.unordered_write: 0 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.row_cache: None 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.wal_filter: None 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T18:27:40.011 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.two_write_queues: 0 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.wal_compression: 0 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.atomic_flush: 0 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.log_readahead_size: 0 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T18:27:40.012 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T18:27:40.014 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_background_jobs: 2 2024-04-23T18:27:40.014 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_background_compactions: -1 2024-04-23T18:27:40.014 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_subcompactions: 1 2024-04-23T18:27:40.014 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T18:27:40.014 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T18:27:40.014 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_open_files: -1 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_background_flushes: -1 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Compression algorithms supported: 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: kZSTD supported: 0 2024-04-23T18:27:40.015 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: kXpressCompression supported: 0 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: kZlibCompression supported: 1 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: kSnappyCompression supported: 1 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: kLZ4Compression supported: 1 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: kBZip2Compression supported: 0 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.merge_operator: 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_filter: None 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T18:27:40.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d7ae905dc0) 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout: cache_index_and_filter_blocks: 1 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout: pin_top_level_index_and_filter: 1 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_type: 0 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout: data_block_index_type: 0 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_shortening: 1 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout: checksum: 4 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout: no_block_cache: 0 2024-04-23T18:27:40.017 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache: 0x55d7ae8cb1f0 2024-04-23T18:27:40.018 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_name: BinnedLRUCache 2024-04-23T18:27:40.018 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_options: 2024-04-23T18:27:40.018 INFO:journalctl@ceph.mon.c.smithi157.stdout: capacity : 536870912 2024-04-23T18:27:40.018 INFO:journalctl@ceph.mon.c.smithi157.stdout: num_shard_bits : 4 2024-04-23T18:27:40.018 INFO:journalctl@ceph.mon.c.smithi157.stdout: strict_capacity_limit : 0 2024-04-23T18:27:40.018 INFO:journalctl@ceph.mon.c.smithi157.stdout: high_pri_pool_ratio: 0.000 2024-04-23T18:27:40.018 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_compressed: (nil) 2024-04-23T18:27:40.018 INFO:journalctl@ceph.mon.c.smithi157.stdout: persistent_cache: (nil) 2024-04-23T18:27:40.018 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_size: 4096 2024-04-23T18:27:40.018 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_size_deviation: 10 2024-04-23T18:27:40.018 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_restart_interval: 16 2024-04-23T18:27:40.018 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_block_restart_interval: 1 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: metadata_block_size: 4096 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: partition_filters: 0 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: use_delta_encoding: 1 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: filter_policy: bloomfilter 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: whole_key_filtering: 1 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: verify_compression: 0 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: read_amp_bytes_per_bit: 0 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: format_version: 5 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: enable_index_compression: 1 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_align: 0 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: max_auto_readahead_size: 262144 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: prepopulate_block_cache: 0 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: initial_auto_readahead_size: 8192 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T18:27:40.019 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compression: NoCompression 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.num_levels: 7 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T18:27:40.020 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:27:40.021 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T18:27:40.022 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T18:27:40.023 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.inplace_update_support: 0 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.bloom_locality: 0 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.max_successive_merges: 0 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.ttl: 2592000 2024-04-23T18:27:40.024 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.enable_blob_files: false 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.min_blob_size: 0 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-23T18:27:40.025 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: eb7edcf4-2033-472d-aa4a-e6c06b4ddd1d 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896859630387, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896859630901, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713896859, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "eb7edcf4-2033-472d-aa4a-e6c06b4ddd1d", "db_session_id": "V6VP358YE6ESVJADT2P3", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896859631002, "job": 1, "event": "recovery_finished"} 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d7ae9e0000 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: DB pointer 0x55d7ae9ca000 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** DB Stats ** 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:27:40.026 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** Compaction Stats [default] ** 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** Compaction Stats [default] ** 2024-04-23T18:27:40.027 INFO:journalctl@ceph.mon.c.smithi157.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T18:27:40.028 INFO:journalctl@ceph.mon.c.smithi157.stdout: Block cache BinnedLRUCache@0x55d7ae8cb1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: using public_addr v2:172.21.15.157:0/0 -> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: starting mon.c rank -1 at public addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] at bind addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(???) e0 preinit fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).mds e1 new map 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout: e1 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout: legacy client fscid: -1 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-04-23T18:27:40.029 INFO:journalctl@ceph.mon.c.smithi157.stdout: No filesystems configured 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:40.030 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: Deploying cephadm binary to smithi157 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:40.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: Added host smithi157 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1786483791' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1786483791' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T18:27:40.032 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.033 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi132:172.21.15.132=a;smithi148:172.21.15.148=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: Saving service mon spec with placement smithi132:172.21.15.132=a;smithi148:172.21.15.148=b;smithi157:172.21.15.157=c;count:3 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: Deploying daemon mon.c on smithi157 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:40.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:40.035 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:39 smithi157 ceph-mon[32580]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-23T18:27:40.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:40 smithi157 bash[32549]: ff4c6dfe43a861932405403ca679b9ce7acadf0fe77d2be10a7e5d07eb678b1b 2024-04-23T18:27:40.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:40 smithi157 systemd[1]: Started Ceph mon.c for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:27:41.194 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T18:27:41.195 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T18:27:41.440 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: mon.a calling monitor election 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: mon.c calling monitor election 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:27:45.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: monmap epoch 2 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: last_changed 2024-04-23T18:27:39.654046+0000 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: created 2024-04-23T18:25:57.540011+0000 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: min_mon_release 19 (squid) 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: election_strategy: 1 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: fsmap 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: mgrmap e13: x(active, since 50s) 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: overall HEALTH_OK 2024-04-23T18:27:45.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:45.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:45.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:27:45.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: mon.a calling monitor election 2024-04-23T18:27:45.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: mon.c calling monitor election 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: monmap epoch 2 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: last_changed 2024-04-23T18:27:39.654046+0000 2024-04-23T18:27:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: created 2024-04-23T18:25:57.540011+0000 2024-04-23T18:27:45.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: min_mon_release 19 (squid) 2024-04-23T18:27:45.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: election_strategy: 1 2024-04-23T18:27:45.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-04-23T18:27:45.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-23T18:27:45.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: fsmap 2024-04-23T18:27:45.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:27:45.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: mgrmap e13: x(active, since 50s) 2024-04-23T18:27:45.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: overall HEALTH_OK 2024-04-23T18:27:45.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:45.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:45.830 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.837+0000 7f6e686a9700 1 -- 172.21.15.157:0/3114564980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e4c0035c0 msgr2=0x7f6e4c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:45.830 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.837+0000 7f6e686a9700 1 --2- 172.21.15.157:0/3114564980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e4c0035c0 0x7f6e4c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6e60079c80 tx=0x7f6e5000feb0 comp rx=0 tx=0).stop 2024-04-23T18:27:45.830 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.838+0000 7f6e686a9700 1 -- 172.21.15.157:0/3114564980 shutdown_connections 2024-04-23T18:27:45.830 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.838+0000 7f6e686a9700 1 --2- 172.21.15.157:0/3114564980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e4c0035c0 0x7f6e4c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:45.831 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.838+0000 7f6e686a9700 1 -- 172.21.15.157:0/3114564980 >> 172.21.15.157:0/3114564980 conn(0x7f6e60077af0 msgr2=0x7f6e60077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:45.831 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.838+0000 7f6e686a9700 1 -- 172.21.15.157:0/3114564980 shutdown_connections 2024-04-23T18:27:45.831 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.838+0000 7f6e686a9700 1 -- 172.21.15.157:0/3114564980 wait complete. 2024-04-23T18:27:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.838+0000 7f6e686a9700 1 Processor -- start 2024-04-23T18:27:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e686a9700 1 -- start start 2024-04-23T18:27:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e686a9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e4c0035c0 0x7f6e60197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e686a9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e601984b0 0x7f6e6019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e686a9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010a4b0 con 0x7f6e601984b0 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e686a9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010a330 con 0x7f6e4c0035c0 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e66445700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e4c0035c0 0x7f6e60197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e66445700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e4c0035c0 0x7f6e60197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46096/0 (socket says 172.21.15.157:46096) 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e66445700 1 -- 172.21.15.157:0/3552397511 learned_addr learned my addr 172.21.15.157:0/3552397511 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e65c44700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e601984b0 0x7f6e6019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e66445700 1 -- 172.21.15.157:0/3552397511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e4c0035c0 msgr2=0x7f6e60197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e66445700 1 -- 172.21.15.157:0/3552397511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e4c0035c0 msgr2=0x7f6e60197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e66445700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e4c0035c0 0x7f6e60197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e66445700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e4c0035c0 0x7f6e60197f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e65c44700 1 -- 172.21.15.157:0/3552397511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e4c0035c0 msgr2=0x7f6e60197f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:27:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e65c44700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e4c0035c0 0x7f6e60197f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:45.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.839+0000 7f6e65c44700 1 -- 172.21.15.157:0/3552397511 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e60198af0 con 0x7f6e601984b0 2024-04-23T18:27:45.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.840+0000 7f6e65c44700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e601984b0 0x7f6e6019c840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00b4d0 tx=0x7f6e5c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:45.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.840+0000 7f6e577fe700 1 -- 172.21.15.157:0/3552397511 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6e5c011630 con 0x7f6e601984b0 2024-04-23T18:27:45.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.840+0000 7f6e686a9700 1 -- 172.21.15.157:0/3552397511 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6019cd80 con 0x7f6e601984b0 2024-04-23T18:27:45.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.840+0000 7f6e577fe700 1 -- 172.21.15.157:0/3552397511 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e5c011c80 con 0x7f6e601984b0 2024-04-23T18:27:45.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.840+0000 7f6e577fe700 1 -- 172.21.15.157:0/3552397511 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6e5c010da0 con 0x7f6e601984b0 2024-04-23T18:27:45.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.840+0000 7f6e686a9700 1 -- 172.21.15.157:0/3552397511 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6019d360 con 0x7f6e601984b0 2024-04-23T18:27:45.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.841+0000 7f6e577fe700 1 -- 172.21.15.157:0/3552397511 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f6e5c018020 con 0x7f6e601984b0 2024-04-23T18:27:45.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.841+0000 7f6e686a9700 1 -- 172.21.15.157:0/3552397511 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6019d670 con 0x7f6e601984b0 2024-04-23T18:27:45.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.841+0000 7f6e577fe700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6e440393f0 0x7f6e4403b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:45.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.841+0000 7f6e577fe700 1 -- 172.21.15.157:0/3552397511 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6e5c04c2a0 con 0x7f6e601984b0 2024-04-23T18:27:45.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.842+0000 7f6e66445700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6e440393f0 0x7f6e4403b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:45.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.842+0000 7f6e66445700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6e440393f0 0x7f6e4403b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000fa60 tx=0x7f6e50005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:45.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:45.847+0000 7f6e577fe700 1 -- 172.21.15.157:0/3552397511 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e6019d670 con 0x7f6e601984b0 2024-04-23T18:27:46.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:45 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:46.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:45 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:27:46.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:45 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:46.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:45 smithi157 ceph-mon[32580]: Deploying daemon mon.b on smithi148 2024-04-23T18:27:46.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:45 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:46.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:45 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:46.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:45 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:27:46.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:45 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:46.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:45 smithi132 ceph-mon[29178]: Deploying daemon mon.b on smithi148 2024-04-23T18:27:46.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:45 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:46.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.129+0000 7f6e686a9700 1 -- 172.21.15.157:0/3552397511 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6e601992d0 con 0x7f6e601984b0 2024-04-23T18:27:46.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.129+0000 7f6e577fe700 1 -- 172.21.15.157:0/3552397511 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f6e5c01f020 con 0x7f6e601984b0 2024-04-23T18:27:46.122 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:27:46.122 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","modified":"2024-04-23T18:27:39.654046Z","created":"2024-04-23T18:25:57.540011Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:3300","nonce":0},{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T18:27:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.131+0000 7f6e686a9700 1 -- 172.21.15.157:0/3552397511 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6e440393f0 msgr2=0x7f6e4403b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.131+0000 7f6e686a9700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6e440393f0 0x7f6e4403b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000fa60 tx=0x7f6e50005960 comp rx=0 tx=0).stop 2024-04-23T18:27:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.132+0000 7f6e686a9700 1 -- 172.21.15.157:0/3552397511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e601984b0 msgr2=0x7f6e6019c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.132+0000 7f6e686a9700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e601984b0 0x7f6e6019c840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00b4d0 tx=0x7f6e5c00b890 comp rx=0 tx=0).stop 2024-04-23T18:27:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.132+0000 7f6e686a9700 1 -- 172.21.15.157:0/3552397511 shutdown_connections 2024-04-23T18:27:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.132+0000 7f6e686a9700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e4c0035c0 0x7f6e60197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.132+0000 7f6e686a9700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6e440393f0 0x7f6e4403b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.132+0000 7f6e686a9700 1 --2- 172.21.15.157:0/3552397511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e601984b0 0x7f6e6019c840 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.132+0000 7f6e686a9700 1 -- 172.21.15.157:0/3552397511 >> 172.21.15.157:0/3552397511 conn(0x7f6e60077af0 msgr2=0x7f6e600fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.132+0000 7f6e686a9700 1 -- 172.21.15.157:0/3552397511 shutdown_connections 2024-04-23T18:27:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:46.132+0000 7f6e686a9700 1 -- 172.21.15.157:0/3552397511 wait complete. 2024-04-23T18:27:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-04-23T18:27:47.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:46 smithi157 ceph-mon[32580]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:47.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:46 smithi157 ceph-mon[32580]: from='client.? 172.21.15.157:0/3552397511' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:27:47.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:46 smithi132 ceph-mon[29178]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:47.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:46 smithi132 ceph-mon[29178]: from='client.? 172.21.15.157:0/3552397511' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:27:47.697 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T18:27:47.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T18:27:47.942 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:27:48.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:48 smithi148 systemd[1]: Started Ceph mon.b for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:27:53.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.838+0000 7f5319d9e700 1 -- 172.21.15.157:0/2628789140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53040057d0 msgr2=0x7f5304007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:53.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.838+0000 7f5319d9e700 1 --2- 172.21.15.157:0/2628789140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53040057d0 0x7f5304007c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5314101190 tx=0x7f530800fe60 comp rx=0 tx=0).stop 2024-04-23T18:27:53.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.838+0000 7f5319d9e700 1 -- 172.21.15.157:0/2628789140 shutdown_connections 2024-04-23T18:27:53.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.838+0000 7f5319d9e700 1 --2- 172.21.15.157:0/2628789140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53040057d0 0x7f5304007c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:53.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.838+0000 7f5319d9e700 1 -- 172.21.15.157:0/2628789140 >> 172.21.15.157:0/2628789140 conn(0x7f53140fcd70 msgr2=0x7f53140ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:53.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.838+0000 7f5319d9e700 1 -- 172.21.15.157:0/2628789140 shutdown_connections 2024-04-23T18:27:53.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.838+0000 7f5319d9e700 1 -- 172.21.15.157:0/2628789140 wait complete. 2024-04-23T18:27:53.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.839+0000 7f5319d9e700 1 Processor -- start 2024-04-23T18:27:53.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.839+0000 7f5319d9e700 1 -- start start 2024-04-23T18:27:53.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.839+0000 7f5319d9e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53040057d0 0x7f531419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:53.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.839+0000 7f5319d9e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531419a700 0x7f531419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:53.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.839+0000 7f5319d9e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f531419ef80 0x7f531419f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:53.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.839+0000 7f5319d9e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5314108140 con 0x7f531419ef80 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.839+0000 7f5319d9e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f531406bae0 con 0x7f531419a700 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.839+0000 7f5319d9e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53141082c0 con 0x7f53040057d0 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f531959d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f531419ef80 0x7f531419f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5318d9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53040057d0 0x7f531419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f531959d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f531419ef80 0x7f531419f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.157:58762/0 (socket says 172.21.15.157:58762) 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f531959d700 1 -- 172.21.15.157:0/3819137496 learned_addr learned my addr 172.21.15.157:0/3819137496 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5318d9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53040057d0 0x7f531419a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51582/0 (socket says 172.21.15.157:51582) 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5313fff700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531419a700 0x7f531419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5318d9c700 1 -- 172.21.15.157:0/3819137496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53040057d0 msgr2=0x7f531419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5318d9c700 1 -- 172.21.15.157:0/3819137496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53040057d0 msgr2=0x7f531419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5318d9c700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53040057d0 0x7f531419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5318d9c700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53040057d0 0x7f531419a1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5313fff700 1 -- 172.21.15.157:0/3819137496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531419a700 msgr2=0x7f531419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5313fff700 1 -- 172.21.15.157:0/3819137496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531419a700 msgr2=0x7f531419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T18:27:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5313fff700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531419a700 0x7f531419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5313fff700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531419a700 0x7f531419ea40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f531959d700 1 -- 172.21.15.157:0/3819137496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53040057d0 msgr2=0x7f531419a1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f531959d700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53040057d0 0x7f531419a1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f531959d700 1 -- 172.21.15.157:0/3819137496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531419a700 msgr2=0x7f531419ea40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f531959d700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531419a700 0x7f531419ea40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f531959d700 1 -- 172.21.15.157:0/3819137496 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53141a6cf0 con 0x7f531419ef80 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f531959d700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f531419ef80 0x7f531419f430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f530c00ba50 tx=0x7f530c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5311ffb700 1 -- 172.21.15.157:0/3819137496 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c019070 con 0x7f531419ef80 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.840+0000 7f5311ffb700 1 -- 172.21.15.157:0/3819137496 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f530c014440 con 0x7f531419ef80 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.841+0000 7f5319d9e700 1 -- 172.21.15.157:0/3819137496 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53141a6f20 con 0x7f531419ef80 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.841+0000 7f5311ffb700 1 -- 172.21.15.157:0/3819137496 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c007590 con 0x7f531419ef80 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.841+0000 7f5319d9e700 1 -- 172.21.15.157:0/3819137496 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53141a7400 con 0x7f531419ef80 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.842+0000 7f5311ffb700 1 -- 172.21.15.157:0/3819137496 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f530c01c070 con 0x7f531419ef80 2024-04-23T18:27:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.842+0000 7f5311ffb700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52f8039ae0 0x7f52f803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:53.839 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.842+0000 7f5311ffb700 1 -- 172.21.15.157:0/3819137496 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f530c04bec0 con 0x7f531419ef80 2024-04-23T18:27:53.839 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.842+0000 7f5319d9e700 1 -- 172.21.15.157:0/3819137496 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53141029a0 con 0x7f531419ef80 2024-04-23T18:27:53.839 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.842+0000 7f5318d9c700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52f8039ae0 0x7f52f803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:53.839 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.843+0000 7f5318d9c700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52f8039ae0 0x7f52f803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f530800fa10 tx=0x7f5308005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:53.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:53.847+0000 7f5311ffb700 1 -- 172.21.15.157:0/3819137496 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f530c00e0a0 con 0x7f531419ef80 2024-04-23T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: mon.a calling monitor election 2024-04-23T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: mon.c calling monitor election 2024-04-23T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: monmap epoch 3 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: last_changed 2024-04-23T18:27:48.586720+0000 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: created 2024-04-23T18:25:57.540011+0000 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: min_mon_release 19 (squid) 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: election_strategy: 1 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: fsmap 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: mgrmap e13: x(active, since 59s) 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: overall HEALTH_OK 2024-04-23T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:54.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:54.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:54.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:54.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: mon.a calling monitor election 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: mon.c calling monitor election 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: monmap epoch 3 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:54.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: last_changed 2024-04-23T18:27:48.586720+0000 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: created 2024-04-23T18:25:57.540011+0000 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: min_mon_release 19 (squid) 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: election_strategy: 1 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: fsmap 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: mgrmap e13: x(active, since 59s) 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: overall HEALTH_OK 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:54.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:54.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.144+0000 7f5319d9e700 1 -- 172.21.15.157:0/3819137496 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f53141a7750 con 0x7f531419ef80 2024-04-23T18:27:54.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.145+0000 7f5311ffb700 1 -- 172.21.15.157:0/3819137496 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f530c014910 con 0x7f531419ef80 2024-04-23T18:27:54.139 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-23T18:27:54.139 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","modified":"2024-04-23T18:27:48.586720Z","created":"2024-04-23T18:25:57.540011Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:3300","nonce":0},{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T18:27:54.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.147+0000 7f5319d9e700 1 -- 172.21.15.157:0/3819137496 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52f8039ae0 msgr2=0x7f52f803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:54.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.147+0000 7f5319d9e700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52f8039ae0 0x7f52f803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f530800fa10 tx=0x7f5308005a90 comp rx=0 tx=0).stop 2024-04-23T18:27:54.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.147+0000 7f5319d9e700 1 -- 172.21.15.157:0/3819137496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f531419ef80 msgr2=0x7f531419f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:54.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.147+0000 7f5319d9e700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f531419ef80 0x7f531419f430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f530c00ba50 tx=0x7f530c00be10 comp rx=0 tx=0).stop 2024-04-23T18:27:54.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.148+0000 7f5319d9e700 1 -- 172.21.15.157:0/3819137496 shutdown_connections 2024-04-23T18:27:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.148+0000 7f5319d9e700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52f8039ae0 0x7f52f803bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.148+0000 7f5319d9e700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f531419ef80 0x7f531419f430 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.148+0000 7f5319d9e700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53040057d0 0x7f531419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.148+0000 7f5319d9e700 1 --2- 172.21.15.157:0/3819137496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531419a700 0x7f531419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.148+0000 7f5319d9e700 1 -- 172.21.15.157:0/3819137496 >> 172.21.15.157:0/3819137496 conn(0x7f53140fcd70 msgr2=0x7f5314104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.148+0000 7f5319d9e700 1 -- 172.21.15.157:0/3819137496 shutdown_connections 2024-04-23T18:27:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:27:54.148+0000 7f5319d9e700 1 -- 172.21.15.157:0/3819137496 wait complete. 2024-04-23T18:27:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 3 2024-04-23T18:27:54.663 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-23T18:27:54.664 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-23T18:27:54.908 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:27:55.014 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:55.014 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:54 smithi157 ceph-mon[32580]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:55.014 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:55.014 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:54 smithi157 ceph-mon[32580]: from='client.? 172.21.15.157:0/3819137496' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:27:55.014 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:55.019 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:55.019 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:54 smithi132 ceph-mon[29178]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:55.020 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:55.020 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:54 smithi132 ceph-mon[29178]: from='client.? 172.21.15.157:0/3819137496' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:27:55.020 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:56.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:27:56.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:56.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: mon.a calling monitor election 2024-04-23T18:27:56.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: mon.c calling monitor election 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: monmap epoch 3 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: last_changed 2024-04-23T18:27:48.586720+0000 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: created 2024-04-23T18:25:57.540011+0000 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: min_mon_release 19 (squid) 2024-04-23T18:27:56.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: election_strategy: 1 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: fsmap 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: mgrmap e13: x(active, since 59s) 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: overall HEALTH_OK 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:56.017 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:56.018 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.018 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='client.? 172.21.15.157:0/3819137496' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:27:56.018 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:56.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.516+0000 7f76460c3700 1 -- 172.21.15.132:0/4266652955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76401073b0 msgr2=0x7f7640107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.516+0000 7f76460c3700 1 --2- 172.21.15.132:0/4266652955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76401073b0 0x7f7640107790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7634009910 tx=0x7f7634009c20 comp rx=0 tx=0).stop 2024-04-23T18:27:56.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.516+0000 7f76460c3700 1 -- 172.21.15.132:0/4266652955 shutdown_connections 2024-04-23T18:27:56.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.516+0000 7f76460c3700 1 --2- 172.21.15.132:0/4266652955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76401073b0 0x7f7640107790 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.516+0000 7f76460c3700 1 -- 172.21.15.132:0/4266652955 >> 172.21.15.132:0/4266652955 conn(0x7f7640077a30 msgr2=0x7f7640077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.516+0000 7f76460c3700 1 -- 172.21.15.132:0/4266652955 shutdown_connections 2024-04-23T18:27:56.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.516+0000 7f76460c3700 1 -- 172.21.15.132:0/4266652955 wait complete. 2024-04-23T18:27:56.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.517+0000 7f76460c3700 1 Processor -- start 2024-04-23T18:27:56.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.517+0000 7f76460c3700 1 -- start start 2024-04-23T18:27:56.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.517+0000 7f76460c3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76401073b0 0x7f7640198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.517+0000 7f76460c3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7640198660 0x7f764019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.517+0000 7f76460c3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764019cf30 0x7f764019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.517+0000 7f76460c3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f764010b810 con 0x7f7640198660 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.517+0000 7f76460c3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f764010b690 con 0x7f76401073b0 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.517+0000 7f76460c3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f764010b990 con 0x7f764019cf30 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7640198660 0x7f764019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7640198660 0x7f764019c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46198/0 (socket says 172.21.15.132:46198) 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763effd700 1 -- 172.21.15.132:0/1529702055 learned_addr learned my addr 172.21.15.132:0/1529702055 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763f7fe700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76401073b0 0x7f7640198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763ffff700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764019cf30 0x7f764019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763effd700 1 -- 172.21.15.132:0/1529702055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764019cf30 msgr2=0x7f764019d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763effd700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764019cf30 0x7f764019d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763effd700 1 -- 172.21.15.132:0/1529702055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76401073b0 msgr2=0x7f7640198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763effd700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76401073b0 0x7f7640198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763effd700 1 -- 172.21.15.132:0/1529702055 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76401a4ca0 con 0x7f7640198660 2024-04-23T18:27:56.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763effd700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7640198660 0x7f764019c9f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f763000b780 tx=0x7f763000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763cff9700 1 -- 172.21.15.132:0/1529702055 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630016070 con 0x7f7640198660 2024-04-23T18:27:56.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763cff9700 1 -- 172.21.15.132:0/1529702055 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76300036a0 con 0x7f7640198660 2024-04-23T18:27:56.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f76460c3700 1 -- 172.21.15.132:0/1529702055 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76401a4ed0 con 0x7f7640198660 2024-04-23T18:27:56.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.518+0000 7f763cff9700 1 -- 172.21.15.132:0/1529702055 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76300119c0 con 0x7f7640198660 2024-04-23T18:27:56.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.519+0000 7f76460c3700 1 -- 172.21.15.132:0/1529702055 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76401a5380 con 0x7f7640198660 2024-04-23T18:27:56.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.519+0000 7f763cff9700 1 -- 172.21.15.132:0/1529702055 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f7630019070 con 0x7f7640198660 2024-04-23T18:27:56.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.519+0000 7f76460c3700 1 -- 172.21.15.132:0/1529702055 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76400640a0 con 0x7f7640198660 2024-04-23T18:27:56.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.520+0000 7f763cff9700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7628039ae0 0x7f762803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.520+0000 7f763cff9700 1 -- 172.21.15.132:0/1529702055 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f763004b300 con 0x7f7640198660 2024-04-23T18:27:56.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.520+0000 7f763f7fe700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7628039ae0 0x7f762803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.520+0000 7f763f7fe700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7628039ae0 0x7f762803bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7634006010 tx=0x7f763401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.529+0000 7f763cff9700 1 -- 172.21.15.132:0/1529702055 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7630015170 con 0x7f7640198660 2024-04-23T18:27:56.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: mon.b calling monitor election 2024-04-23T18:27:56.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: Updating smithi132:/etc/ceph/ceph.conf 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: mon.b calling monitor election 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: mon.c calling monitor election 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: mon.a calling monitor election 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: monmap epoch 3 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: last_changed 2024-04-23T18:27:48.586720+0000 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: created 2024-04-23T18:25:57.540011+0000 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: min_mon_release 19 (squid) 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: election_strategy: 1 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-23T18:27:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: fsmap 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: mgrmap e13: x(active, since 61s) 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: overall HEALTH_OK 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: Updating smithi132:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:56.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T18:27:56.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:27:56.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:27:56.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:56.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:56 smithi132 ceph-mon[29178]: Reconfiguring daemon mon.a on smithi132 2024-04-23T18:27:56.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.736+0000 7f76460c3700 1 -- 172.21.15.132:0/1529702055 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f76401a56f0 con 0x7f7640198660 2024-04-23T18:27:56.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.736+0000 7f763cff9700 1 -- 172.21.15.132:0/1529702055 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f7630003850 con 0x7f7640198660 2024-04-23T18:27:56.730 INFO:teuthology.orchestra.run.smithi132.stdout:# minimal ceph.conf for c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:56.730 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2024-04-23T18:27:56.730 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:56.730 INFO:teuthology.orchestra.run.smithi132.stdout: mon_host = [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-04-23T18:27:56.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.739+0000 7f76460c3700 1 -- 172.21.15.132:0/1529702055 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7628039ae0 msgr2=0x7f762803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.739+0000 7f76460c3700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7628039ae0 0x7f762803bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7634006010 tx=0x7f763401a040 comp rx=0 tx=0).stop 2024-04-23T18:27:56.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.739+0000 7f76460c3700 1 -- 172.21.15.132:0/1529702055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7640198660 msgr2=0x7f764019c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.739+0000 7f76460c3700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7640198660 0x7f764019c9f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f763000b780 tx=0x7f763000ba90 comp rx=0 tx=0).stop 2024-04-23T18:27:56.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.739+0000 7f76460c3700 1 -- 172.21.15.132:0/1529702055 shutdown_connections 2024-04-23T18:27:56.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.739+0000 7f76460c3700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764019cf30 0x7f764019d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.739+0000 7f76460c3700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76401073b0 0x7f7640198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.739+0000 7f76460c3700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7628039ae0 0x7f762803bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.739+0000 7f76460c3700 1 --2- 172.21.15.132:0/1529702055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7640198660 0x7f764019c9f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.739+0000 7f76460c3700 1 -- 172.21.15.132:0/1529702055 >> 172.21.15.132:0/1529702055 conn(0x7f7640077a30 msgr2=0x7f7640190ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.739+0000 7f76460c3700 1 -- 172.21.15.132:0/1529702055 shutdown_connections 2024-04-23T18:27:56.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:27:56.739+0000 7f76460c3700 1 -- 172.21.15.132:0/1529702055 wait complete. 2024-04-23T18:27:56.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: mon.b calling monitor election 2024-04-23T18:27:56.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: Updating smithi132:/etc/ceph/ceph.conf 2024-04-23T18:27:56.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-23T18:27:56.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-23T18:27:56.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: mon.b calling monitor election 2024-04-23T18:27:56.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: mon.c calling monitor election 2024-04-23T18:27:56.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: mon.a calling monitor election 2024-04-23T18:27:56.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T18:27:56.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: monmap epoch 3 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: last_changed 2024-04-23T18:27:48.586720+0000 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: created 2024-04-23T18:25:57.540011+0000 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: min_mon_release 19 (squid) 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: election_strategy: 1 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: fsmap 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: mgrmap e13: x(active, since 61s) 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: overall HEALTH_OK 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: Updating smithi132:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:56.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:27:56.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:27:56.768 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:56.768 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:56 smithi148 ceph-mon[32311]: Reconfiguring daemon mon.a on smithi132 2024-04-23T18:27:56.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: mon.b calling monitor election 2024-04-23T18:27:56.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: Updating smithi132:/etc/ceph/ceph.conf 2024-04-23T18:27:56.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: mon.b calling monitor election 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: mon.c calling monitor election 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: mon.a calling monitor election 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: monmap epoch 3 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: last_changed 2024-04-23T18:27:48.586720+0000 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: created 2024-04-23T18:25:57.540011+0000 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: min_mon_release 19 (squid) 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: election_strategy: 1 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-04-23T18:27:56.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: fsmap 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: mgrmap e13: x(active, since 61s) 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: overall HEALTH_OK 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: Updating smithi132:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:56.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.793 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.793 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.793 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:56.793 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T18:27:56.793 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:27:56.793 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:27:56.793 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:56.793 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:56 smithi157 ceph-mon[32580]: Reconfiguring daemon mon.a on smithi132 2024-04-23T18:27:57.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:57.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:57 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1529702055' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:57.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:57.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:57 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1529702055' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:57.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:27:57.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:57 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1529702055' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:58.259 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-23T18:27:58.259 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:27:58.259 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T18:27:58.289 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:27:58.289 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:27:58.354 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T18:27:58.354 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T18:27:58.380 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T18:27:58.380 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:27:58.446 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-23T18:27:58.446 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T18:27:58.479 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-23T18:27:58.479 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:27:58.547 INFO:tasks.cephadm:Adding mgr.x on smithi132 2024-04-23T18:27:58.547 INFO:tasks.cephadm:Adding mgr.y on smithi148 2024-04-23T18:27:58.547 INFO:tasks.cephadm:Adding mgr.z on smithi157 2024-04-23T18:27:58.547 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi132=x;smithi148=y;smithi157=z' 2024-04-23T18:27:58.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:58.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:58.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:58 smithi157 ceph-mon[32580]: Reconfiguring mon.b (monmap changed)... 2024-04-23T18:27:58.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:27:58.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:27:58.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:58.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:58 smithi157 ceph-mon[32580]: Reconfiguring daemon mon.b on smithi148 2024-04-23T18:27:58.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:27:58 smithi157 ceph-mon[32580]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:58.823 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:27:59.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:59.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:59.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:58 smithi148 ceph-mon[32311]: Reconfiguring mon.b (monmap changed)... 2024-04-23T18:27:59.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:27:59.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:27:59.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:59.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:58 smithi148 ceph-mon[32311]: Reconfiguring daemon mon.b on smithi148 2024-04-23T18:27:59.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:27:58 smithi148 ceph-mon[32311]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:27:59.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:59.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:27:59.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:58 smithi132 ceph-mon[29178]: Reconfiguring mon.b (monmap changed)... 2024-04-23T18:27:59.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:27:59.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:27:59.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:59.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:58 smithi132 ceph-mon[29178]: Reconfiguring daemon mon.b on smithi148 2024-04-23T18:27:59.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:27:58 smithi132 ceph-mon[29178]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:00.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:00 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:00.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:00 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:00.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:00 smithi148 ceph-mon[32311]: Reconfiguring mon.c (monmap changed)... 2024-04-23T18:28:00.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:00 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:28:00.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:00 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:28:00.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:00 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:00.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:00 smithi148 ceph-mon[32311]: Reconfiguring daemon mon.c on smithi157 2024-04-23T18:28:00.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:00 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:00.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:00 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:00.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:00 smithi157 ceph-mon[32580]: Reconfiguring mon.c (monmap changed)... 2024-04-23T18:28:00.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:00 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:28:00.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:00 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:28:00.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:00 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:00.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:00 smithi157 ceph-mon[32580]: Reconfiguring daemon mon.c on smithi157 2024-04-23T18:28:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:00 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:00 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:00 smithi132 ceph-mon[29178]: Reconfiguring mon.c (monmap changed)... 2024-04-23T18:28:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:00 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:28:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:00 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:28:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:00 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:00 smithi132 ceph-mon[29178]: Reconfiguring daemon mon.c on smithi157 2024-04-23T18:28:01.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.248+0000 7fde31a6d700 1 -- 172.21.15.157:0/474317772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c108790 msgr2=0x7fde2c10ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:01.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.248+0000 7fde31a6d700 1 --2- 172.21.15.157:0/474317772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c108790 0x7fde2c10ac00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fde2400b0a0 tx=0x7fde2400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:28:01.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.248+0000 7fde31a6d700 1 -- 172.21.15.157:0/474317772 shutdown_connections 2024-04-23T18:28:01.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.248+0000 7fde31a6d700 1 --2- 172.21.15.157:0/474317772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c108790 0x7fde2c10ac00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:01.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.248+0000 7fde31a6d700 1 --2- 172.21.15.157:0/474317772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde2c103530 0x7fde2c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:01.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.248+0000 7fde31a6d700 1 --2- 172.21.15.157:0/474317772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c105e60 0x7fde2c108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.248+0000 7fde31a6d700 1 -- 172.21.15.157:0/474317772 >> 172.21.15.157:0/474317772 conn(0x7fde2c0fcf60 msgr2=0x7fde2c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.248+0000 7fde31a6d700 1 -- 172.21.15.157:0/474317772 shutdown_connections 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.248+0000 7fde31a6d700 1 -- 172.21.15.157:0/474317772 wait complete. 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.248+0000 7fde31a6d700 1 Processor -- start 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.248+0000 7fde31a6d700 1 -- start start 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde31a6d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c103530 0x7fde2c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde31a6d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde2c105e60 0x7fde2c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde31a6d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c108790 0x7fde2c1a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde31a6d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c10e570 con 0x7fde2c103530 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde31a6d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c10e3f0 con 0x7fde2c105e60 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde31a6d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c10e6f0 con 0x7fde2c108790 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde2bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c108790 0x7fde2c1a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde2bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c108790 0x7fde2c1a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47706/0 (socket says 172.21.15.157:47706) 2024-04-23T18:28:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde2bfff700 1 -- 172.21.15.157:0/3009179461 learned_addr learned my addr 172.21.15.157:0/3009179461 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde2b7fe700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c103530 0x7fde2c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde22ffd700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde2c105e60 0x7fde2c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde2bfff700 1 -- 172.21.15.157:0/3009179461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde2c105e60 msgr2=0x7fde2c1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde2bfff700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde2c105e60 0x7fde2c1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde2bfff700 1 -- 172.21.15.157:0/3009179461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c103530 msgr2=0x7fde2c198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde2bfff700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c103530 0x7fde2c198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde2bfff700 1 -- 172.21.15.157:0/3009179461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2c1a0810 con 0x7fde2c108790 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde2bfff700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c108790 0x7fde2c1a0170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fde24009be0 tx=0x7fde24004dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde297fa700 1 -- 172.21.15.157:0/3009179461 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24021070 con 0x7fde2c108790 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde297fa700 1 -- 172.21.15.157:0/3009179461 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde24007c00 con 0x7fde2c108790 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.249+0000 7fde31a6d700 1 -- 172.21.15.157:0/3009179461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2c1a0aa0 con 0x7fde2c108790 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.250+0000 7fde297fa700 1 -- 172.21.15.157:0/3009179461 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24004800 con 0x7fde2c108790 2024-04-23T18:28:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.250+0000 7fde31a6d700 1 -- 172.21.15.157:0/3009179461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2c1a0fe0 con 0x7fde2c108790 2024-04-23T18:28:01.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.250+0000 7fde297fa700 1 -- 172.21.15.157:0/3009179461 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fde2401d450 con 0x7fde2c108790 2024-04-23T18:28:01.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.250+0000 7fde31a6d700 1 -- 172.21.15.157:0/3009179461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde2c069730 con 0x7fde2c108790 2024-04-23T18:28:01.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.251+0000 7fde297fa700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fde18039890 0x7fde1803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:01.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.251+0000 7fde297fa700 1 -- 172.21.15.157:0/3009179461 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fde2400f020 con 0x7fde2c108790 2024-04-23T18:28:01.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.251+0000 7fde2b7fe700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fde18039890 0x7fde1803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:01.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.251+0000 7fde2b7fe700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fde18039890 0x7fde1803bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005950 tx=0x7fde1c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:01.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.256+0000 7fde297fa700 1 -- 172.21.15.157:0/3009179461 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde24052070 con 0x7fde2c108790 2024-04-23T18:28:01.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:01 smithi157 ceph-mon[32580]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:01.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:01.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:01.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:01.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.466+0000 7fde31a6d700 1 -- 172.21.15.157:0/3009179461 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi132=x;smithi148=y;smithi157=z", "target": ["mon-mgr", ""]}) -- 0x7fde2c062f10 con 0x7fde18039890 2024-04-23T18:28:01.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.472+0000 7fde297fa700 1 -- 172.21.15.157:0/3009179461 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fde2c062f10 con 0x7fde18039890 2024-04-23T18:28:01.465 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mgr update... 2024-04-23T18:28:01.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.474+0000 7fde31a6d700 1 -- 172.21.15.157:0/3009179461 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fde18039890 msgr2=0x7fde1803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:01.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.474+0000 7fde31a6d700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fde18039890 0x7fde1803bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005950 tx=0x7fde1c00a300 comp rx=0 tx=0).stop 2024-04-23T18:28:01.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.474+0000 7fde31a6d700 1 -- 172.21.15.157:0/3009179461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c108790 msgr2=0x7fde2c1a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:01.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.474+0000 7fde31a6d700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c108790 0x7fde2c1a0170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fde24009be0 tx=0x7fde24004dc0 comp rx=0 tx=0).stop 2024-04-23T18:28:01.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.474+0000 7fde31a6d700 1 -- 172.21.15.157:0/3009179461 shutdown_connections 2024-04-23T18:28:01.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.474+0000 7fde31a6d700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c108790 0x7fde2c1a0170 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:01.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.474+0000 7fde31a6d700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fde18039890 0x7fde1803bd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:01.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.474+0000 7fde31a6d700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c103530 0x7fde2c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:01.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.474+0000 7fde31a6d700 1 --2- 172.21.15.157:0/3009179461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde2c105e60 0x7fde2c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:01.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.474+0000 7fde31a6d700 1 -- 172.21.15.157:0/3009179461 >> 172.21.15.157:0/3009179461 conn(0x7fde2c0fcf60 msgr2=0x7fde2c103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:01.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.474+0000 7fde31a6d700 1 -- 172.21.15.157:0/3009179461 shutdown_connections 2024-04-23T18:28:01.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:01.474+0000 7fde31a6d700 1 -- 172.21.15.157:0/3009179461 wait complete. 2024-04-23T18:28:01.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:01 smithi148 ceph-mon[32311]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:01.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:01.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:01.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:01.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:01 smithi132 ceph-mon[29178]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:01.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:01.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:01.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:02.092 DEBUG:teuthology.orchestra.run.smithi148:mgr.y> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.y.service 2024-04-23T18:28:02.095 DEBUG:teuthology.orchestra.run.smithi157:mgr.z> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.z.service 2024-04-23T18:28:02.097 INFO:tasks.cephadm:Deploying OSDs... 2024-04-23T18:28:02.097 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:28:02.097 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T18:28:02.114 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T18:28:02.115 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2024-04-23T18:28:02.169 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T18:28:02.170 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:28:02.170 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-23T18:28:02.170 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:28:02.170 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:28:02.170 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-23 18:27:06.708835328 +0000 2024-04-23T18:28:02.170 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-23 18:21:29.504842740 +0000 2024-04-23T18:28:02.170 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-23 18:21:29.504842740 +0000 2024-04-23T18:28:02.170 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-23 18:21:29.504842740 +0000 2024-04-23T18:28:02.171 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T18:28:02.233 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-04-23T18:28:02.233 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-04-23T18:28:02.233 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000207953 s, 2.5 MB/s 2024-04-23T18:28:02.234 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T18:28:02.291 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2024-04-23T18:28:02.345 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T18:28:02.345 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:28:02.345 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-23T18:28:02.346 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:28:02.346 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:28:02.346 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-23 18:27:06.734834628 +0000 2024-04-23T18:28:02.346 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-23 18:21:29.834833761 +0000 2024-04-23T18:28:02.346 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-23 18:21:29.834833761 +0000 2024-04-23T18:28:02.346 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-23 18:21:29.834833761 +0000 2024-04-23T18:28:02.346 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T18:28:02.407 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-04-23T18:28:02.407 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-04-23T18:28:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000206337 s, 2.5 MB/s 2024-04-23T18:28:02.409 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T18:28:02.465 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2024-04-23T18:28:02.520 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T18:28:02.520 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:28:02.520 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-23T18:28:02.521 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:28:02.521 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:28:02.521 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-23 18:27:06.759833955 +0000 2024-04-23T18:28:02.521 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-23 18:21:30.164824779 +0000 2024-04-23T18:28:02.521 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-23 18:21:30.164824779 +0000 2024-04-23T18:28:02.521 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-23 18:21:30.164824779 +0000 2024-04-23T18:28:02.521 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T18:28:02.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:02 smithi157 ceph-mon[32580]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi132=x;smithi148=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:28:02.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:02 smithi157 ceph-mon[32580]: Saving service mgr spec with placement smithi132=x;smithi148=y;smithi157=z;count:3 2024-04-23T18:28:02.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:02 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:02.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:02 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:02.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:02 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:02.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:02 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:02.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:02 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:28:02.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:02 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:28:02.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:02 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:28:02.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:02 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:02.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:02 smithi157 ceph-mon[32580]: Deploying daemon mgr.z on smithi157 2024-04-23T18:28:02.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:02 smithi157 ceph-mon[32580]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:02.580 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:02 smithi132 ceph-mon[29178]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi132=x;smithi148=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:28:02.580 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:02 smithi132 ceph-mon[29178]: Saving service mgr spec with placement smithi132=x;smithi148=y;smithi157=z;count:3 2024-04-23T18:28:02.580 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:02 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:02.580 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:02 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:02.581 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:02 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:02.581 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:02 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:02.581 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:02 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:28:02.581 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:02 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:28:02.581 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:02 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:28:02.581 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:02 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:02.581 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:02 smithi132 ceph-mon[29178]: Deploying daemon mgr.z on smithi157 2024-04-23T18:28:02.581 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:02 smithi132 ceph-mon[29178]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:02.582 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-04-23T18:28:02.582 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-04-23T18:28:02.582 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000216593 s, 2.4 MB/s 2024-04-23T18:28:02.583 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T18:28:02.639 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2024-04-23T18:28:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T18:28:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:28:02.694 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-23T18:28:02.694 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:28:02.694 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:28:02.694 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-23 18:27:06.783833308 +0000 2024-04-23T18:28:02.695 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-23 18:21:30.504815522 +0000 2024-04-23T18:28:02.695 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-23 18:21:30.504815522 +0000 2024-04-23T18:28:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-23 18:21:30.504815522 +0000 2024-04-23T18:28:02.695 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T18:28:02.757 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-04-23T18:28:02.758 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-04-23T18:28:02.758 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000203093 s, 2.5 MB/s 2024-04-23T18:28:02.759 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T18:28:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:02 smithi148 ceph-mon[32311]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi132=x;smithi148=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:28:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:02 smithi148 ceph-mon[32311]: Saving service mgr spec with placement smithi132=x;smithi148=y;smithi157=z;count:3 2024-04-23T18:28:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:02 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:02 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:02 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:02 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:02 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:28:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:02 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:28:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:02 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:28:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:02 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:02 smithi148 ceph-mon[32311]: Deploying daemon mgr.z on smithi157 2024-04-23T18:28:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:02 smithi148 ceph-mon[32311]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:02.815 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T18:28:02.815 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T18:28:02.834 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T18:28:02.834 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-04-23T18:28:02.891 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T18:28:02.891 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:28:02.892 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-23T18:28:02.892 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:28:02.892 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:28:02.892 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-23 18:27:48.031288080 +0000 2024-04-23T18:28:02.892 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-23 18:21:57.738693152 +0000 2024-04-23T18:28:02.892 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-23 18:21:57.738693152 +0000 2024-04-23T18:28:02.892 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-23 18:21:57.738693152 +0000 2024-04-23T18:28:02.892 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T18:28:02.958 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-23T18:28:02.958 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-23T18:28:02.959 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000359178 s, 1.4 MB/s 2024-04-23T18:28:02.960 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T18:28:03.018 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-04-23T18:28:03.075 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T18:28:03.075 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:28:03.075 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-23T18:28:03.075 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:28:03.075 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:28:03.075 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-23 18:27:48.032288057 +0000 2024-04-23T18:28:03.075 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-23 18:21:58.089684712 +0000 2024-04-23T18:28:03.075 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-23 18:21:58.089684712 +0000 2024-04-23T18:28:03.076 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-23 18:21:58.089684712 +0000 2024-04-23T18:28:03.076 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T18:28:03.140 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-23T18:28:03.140 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-23T18:28:03.140 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000323344 s, 1.6 MB/s 2024-04-23T18:28:03.141 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T18:28:03.198 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-04-23T18:28:03.256 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T18:28:03.288 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:28:03.289 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-23T18:28:03.291 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:28:03.291 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:28:03.291 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-23 18:27:48.032288057 +0000 2024-04-23T18:28:03.291 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-23 18:21:58.429676537 +0000 2024-04-23T18:28:03.291 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-23 18:21:58.429676537 +0000 2024-04-23T18:28:03.291 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-23 18:21:58.429676537 +0000 2024-04-23T18:28:03.292 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T18:28:03.327 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-23T18:28:03.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-23T18:28:03.328 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000220049 s, 2.3 MB/s 2024-04-23T18:28:03.329 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T18:28:03.388 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-04-23T18:28:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T18:28:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:28:03.445 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-23T18:28:03.445 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:28:03.445 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:28:03.446 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-23 18:27:48.033288033 +0000 2024-04-23T18:28:03.446 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-23 18:21:58.772668289 +0000 2024-04-23T18:28:03.446 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-23 18:21:58.772668289 +0000 2024-04-23T18:28:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-23 18:21:58.772668289 +0000 2024-04-23T18:28:03.446 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T18:28:03.511 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-23T18:28:03.512 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-23T18:28:03.512 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000235563 s, 2.2 MB/s 2024-04-23T18:28:03.514 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T18:28:03.573 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-23T18:28:03.573 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T18:28:03.591 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T18:28:03.591 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-04-23T18:28:03.648 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T18:28:03.648 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:28:03.648 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-23T18:28:03.649 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:28:03.649 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:28:03.649 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-23 18:28:03.403787223 +0000 2024-04-23T18:28:03.649 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-23 18:22:03.511686403 +0000 2024-04-23T18:28:03.649 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-23 18:22:03.511686403 +0000 2024-04-23T18:28:03.650 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-23 18:22:03.511686403 +0000 2024-04-23T18:28:03.650 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T18:28:03.675 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:03 smithi157 systemd[1]: Starting Ceph mgr.z for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:28:03.700 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-23T18:28:03.701 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-23T18:28:03.701 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000112388 s, 4.6 MB/s 2024-04-23T18:28:03.703 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T18:28:03.760 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-04-23T18:28:03.817 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T18:28:03.818 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:28:03.818 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-23T18:28:03.818 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:28:03.818 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:28:03.818 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-23 18:28:03.404787204 +0000 2024-04-23T18:28:03.818 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-23 18:22:03.840680072 +0000 2024-04-23T18:28:03.818 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-23 18:22:03.840680072 +0000 2024-04-23T18:28:03.818 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-23 18:22:03.840680072 +0000 2024-04-23T18:28:03.818 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T18:28:03.881 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-23T18:28:03.881 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-23T18:28:03.881 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000108434 s, 4.7 MB/s 2024-04-23T18:28:03.882 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T18:28:03.929 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:03 smithi157 podman[34189]: 2024-04-23 18:28:03.586669145 +0000 UTC m=+0.018359350 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:28:03.929 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:03 smithi157 podman[34189]: 2024-04-23 18:28:03.707352116 +0000 UTC m=+0.139042304 container create 7f74d05ed853e73da54e37f5a908a6a74f1810cb3047d1d2c11842f5c474b9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-23T18:28:03.929 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:03 smithi157 podman[34189]: 2024-04-23 18:28:03.909769191 +0000 UTC m=+0.341459391 container init 7f74d05ed853e73da54e37f5a908a6a74f1810cb3047d1d2c11842f5c474b9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:28:03.930 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:03 smithi157 podman[34189]: 2024-04-23 18:28:03.912324581 +0000 UTC m=+0.344014771 container start 7f74d05ed853e73da54e37f5a908a6a74f1810cb3047d1d2c11842f5c474b9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-23T18:28:03.939 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-04-23T18:28:03.994 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T18:28:03.994 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:28:03.994 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-23T18:28:03.994 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:28:03.994 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:28:03.994 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-23 18:28:03.404787204 +0000 2024-04-23T18:28:03.994 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-23 18:22:04.181673510 +0000 2024-04-23T18:28:03.994 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-23 18:22:04.181673510 +0000 2024-04-23T18:28:03.994 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-23 18:22:04.181673510 +0000 2024-04-23T18:28:03.995 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T18:28:04.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-23T18:28:04.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-23T18:28:04.059 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.3307e-05 s, 5.5 MB/s 2024-04-23T18:28:04.060 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T18:28:04.117 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-04-23T18:28:04.172 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T18:28:04.172 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:28:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-23T18:28:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:28:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:28:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-23 18:28:03.404787204 +0000 2024-04-23T18:28:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-23 18:22:04.512667141 +0000 2024-04-23T18:28:04.173 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-23 18:22:04.512667141 +0000 2024-04-23T18:28:04.173 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-23 18:22:04.512667141 +0000 2024-04-23T18:28:04.173 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T18:28:04.233 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:03 smithi157 ceph-mgr[34312]: -- 172.21.15.157:0/1982045501 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d2a4de5a0 con 0x555d2a4b2c00 2024-04-23T18:28:04.233 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:04 smithi157 bash[34189]: 7f74d05ed853e73da54e37f5a908a6a74f1810cb3047d1d2c11842f5c474b9ca 2024-04-23T18:28:04.233 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:04 smithi157 systemd[1]: Started Ceph mgr.z for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:28:04.234 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:04.072+0000 7f8be5bfb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:28:04.235 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-23T18:28:04.235 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-23T18:28:04.235 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 8.8772e-05 s, 5.8 MB/s 2024-04-23T18:28:04.237 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T18:28:04.295 INFO:tasks.cephadm:Deploying osd.0 on smithi132 with /dev/vg_nvme/lv_4... 2024-04-23T18:28:04.295 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T18:28:04.537 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:28:04.540 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:04.267+0000 7f8be5bfb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:28:04.540 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:04.378+0000 7f8be5bfb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:28:05.040 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:04.705+0000 7f8be5bfb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:28:05.040 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:04.792+0000 7f8be5bfb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:28:05.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:05 smithi132 ceph-mon[29178]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:05.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:05 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:05.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:05 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:05.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:05 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:05.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:05 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:28:05.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:05 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:28:05.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:05 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:28:05.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:05 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:05.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:05 smithi132 ceph-mon[29178]: Deploying daemon mgr.y on smithi148 2024-04-23T18:28:05.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:05 smithi148 ceph-mon[32311]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:05.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:05 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:05.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:05 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:05.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:05 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:05.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:05 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:28:05.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:05 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:28:05.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:05 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:28:05.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:05 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:05.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:05 smithi148 ceph-mon[32311]: Deploying daemon mgr.y on smithi148 2024-04-23T18:28:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:05 smithi157 ceph-mon[32580]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:05 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:05 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:05 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:05.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:05 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:28:05.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:05 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:28:05.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:05 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:28:05.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:05 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:05.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:05 smithi157 ceph-mon[32580]: Deploying daemon mgr.y on smithi148 2024-04-23T18:28:06.265 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:06 smithi148 systemd[1]: Starting Ceph mgr.y for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:28:06.290 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:05.999+0000 7f8be5bfb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:28:06.290 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:06.167+0000 7f8be5bfb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:28:06.481 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T18:28:06.482 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T18:28:06.482 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-04-23T18:28:06.482 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-04-23T18:28:06.482 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130296 s, 805 MB/s 2024-04-23T18:28:06.482 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-04-23T18:28:06.614 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:06.338+0000 7f8be5bfb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:28:06.614 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:06.620+0000 7f8be5bfb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:28:06.684 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:06 smithi148 podman[33676]: 2024-04-23 18:28:06.237792531 +0000 UTC m=+0.018933335 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:28:06.684 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:06 smithi148 podman[33676]: 2024-04-23 18:28:06.33900829 +0000 UTC m=+0.120149094 container create 203519ea92c8998d25c6af184c8ab0479f15d90e50373982d0f134e0d9054e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-23T18:28:06.685 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:06 smithi148 podman[33676]: 2024-04-23 18:28:06.51651278 +0000 UTC m=+0.297653584 container init 203519ea92c8998d25c6af184c8ab0479f15d90e50373982d0f134e0d9054e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:28:06.685 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:06 smithi148 podman[33676]: 2024-04-23 18:28:06.519590785 +0000 UTC m=+0.300731598 container start 203519ea92c8998d25c6af184c8ab0479f15d90e50373982d0f134e0d9054e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-04-23T18:28:06.685 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:06 smithi148 ceph-mgr[33699]: -- 172.21.15.148:0/4224667095 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a82a9ba5a0 con 0x55a82a98f000 2024-04-23T18:28:06.685 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:06 smithi148 bash[33676]: 203519ea92c8998d25c6af184c8ab0479f15d90e50373982d0f134e0d9054e24 2024-04-23T18:28:06.685 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:06 smithi148 systemd[1]: Started Ceph mgr.y for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:28:06.685 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:06 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:06.690+0000 7fed7f4c5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:28:06.914 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:06.713+0000 7f8be5bfb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:28:06.914 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:06.920+0000 7f8be5bfb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:28:06.990 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:06 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:06.886+0000 7fed7f4c5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:28:06.990 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:06 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:06.996+0000 7fed7f4c5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:28:07.043 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi132:vg_nvme/lv_4 2024-04-23T18:28:07.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:07 smithi148 ceph-mon[32311]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:07.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:07 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:07.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:07 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:07.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:07 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:07.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:07 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:07.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:07 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:07.280 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:28:07.290 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:07 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:07.118+0000 7f8be5bfb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:28:07.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:07 smithi157 ceph-mon[32580]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:07.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:07 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:07.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:07 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:07.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:07 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:07.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:07 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:07.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:07 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:07 smithi132 ceph-mon[29178]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:07 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:07 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:07 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:07 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:07 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:07.674 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:07 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:07.331+0000 7fed7f4c5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:28:07.674 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:07 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:07.442+0000 7fed7f4c5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:28:08.040 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:07 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:07.752+0000 7f8be5bfb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:28:08.480 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:08 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:08.188+0000 7f8be5bfb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:28:08.480 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:08 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:08.284+0000 7f8be5bfb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:28:08.480 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:08 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:08.486+0000 7f8be5bfb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:28:08.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.810+0000 7f8f2fc63700 1 -- 172.21.15.132:0/3793545464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f28101990 msgr2=0x7f8f28101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:08.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.810+0000 7f8f2fc63700 1 --2- 172.21.15.132:0/3793545464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f28101990 0x7f8f28101db0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f240099c0 tx=0x7f8f24009cd0 comp rx=0 tx=0).stop 2024-04-23T18:28:08.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.810+0000 7f8f2fc63700 1 -- 172.21.15.132:0/3793545464 shutdown_connections 2024-04-23T18:28:08.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.810+0000 7f8f2fc63700 1 --2- 172.21.15.132:0/3793545464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28103530 0x7f8f28109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:08.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.810+0000 7f8f2fc63700 1 --2- 172.21.15.132:0/3793545464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f28101990 0x7f8f28101db0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:08.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.810+0000 7f8f2fc63700 1 --2- 172.21.15.132:0/3793545464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f28102b90 0x7f8f28102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:08.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.810+0000 7f8f2fc63700 1 -- 172.21.15.132:0/3793545464 >> 172.21.15.132:0/3793545464 conn(0x7f8f280fcf30 msgr2=0x7f8f280ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:08.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.810+0000 7f8f2fc63700 1 -- 172.21.15.132:0/3793545464 shutdown_connections 2024-04-23T18:28:08.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.810+0000 7f8f2fc63700 1 -- 172.21.15.132:0/3793545464 wait complete. 2024-04-23T18:28:08.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.811+0000 7f8f2fc63700 1 Processor -- start 2024-04-23T18:28:08.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.811+0000 7f8f2fc63700 1 -- start start 2024-04-23T18:28:08.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.811+0000 7f8f2fc63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f28101990 0x7f8f2806bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:08.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.811+0000 7f8f2fc63700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f28102b90 0x7f8f2819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:08.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.811+0000 7f8f2fc63700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28103530 0x7f8f281a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:08.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.811+0000 7f8f2fc63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810d420 con 0x7f8f28101990 2024-04-23T18:28:08.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.811+0000 7f8f2fc63700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810d2a0 con 0x7f8f28103530 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.811+0000 7f8f2fc63700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810d5a0 con 0x7f8f28102b90 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.811+0000 7f8f2d9ff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f28101990 0x7f8f2806bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.811+0000 7f8f2d9ff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f28101990 0x7f8f2806bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42414/0 (socket says 172.21.15.132:42414) 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.811+0000 7f8f2d9ff700 1 -- 172.21.15.132:0/1549788905 learned_addr learned my addr 172.21.15.132:0/1549788905 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.811+0000 7f8f2d1fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f28102b90 0x7f8f2819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f2e200700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28103530 0x7f8f281a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f2d9ff700 1 -- 172.21.15.132:0/1549788905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f28102b90 msgr2=0x7f8f2819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f2d9ff700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f28102b90 0x7f8f2819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f2d9ff700 1 -- 172.21.15.132:0/1549788905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28103530 msgr2=0x7f8f281a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f2d9ff700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28103530 0x7f8f281a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f2d9ff700 1 -- 172.21.15.132:0/1549788905 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f281a2a50 con 0x7f8f28101990 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f2d1fe700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f28102b90 0x7f8f2819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f2d9ff700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f28101990 0x7f8f2806bac0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f240036f0 tx=0x7f8f2400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:08.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f1effd700 1 -- 172.21.15.132:0/1549788905 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24009e60 con 0x7f8f28101990 2024-04-23T18:28:08.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f1effd700 1 -- 172.21.15.132:0/1549788905 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f24005940 con 0x7f8f28101990 2024-04-23T18:28:08.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f2fc63700 1 -- 172.21.15.132:0/1549788905 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f281a2ce0 con 0x7f8f28101990 2024-04-23T18:28:08.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f1effd700 1 -- 172.21.15.132:0/1549788905 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f240054f0 con 0x7f8f28101990 2024-04-23T18:28:08.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.812+0000 7f8f2fc63700 1 -- 172.21.15.132:0/1549788905 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f281a31c0 con 0x7f8f28101990 2024-04-23T18:28:08.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.813+0000 7f8f1effd700 1 -- 172.21.15.132:0/1549788905 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f8f2402b440 con 0x7f8f28101990 2024-04-23T18:28:08.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.813+0000 7f8f1effd700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8f14039890 0x7f8f1403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:08.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.813+0000 7f8f1effd700 1 -- 172.21.15.132:0/1549788905 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8f2404ea40 con 0x7f8f28101990 2024-04-23T18:28:08.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.813+0000 7f8f2d1fe700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8f14039890 0x7f8f1403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:08.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.814+0000 7f8f2fc63700 1 -- 172.21.15.132:0/1549788905 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2802df80 con 0x7f8f28101990 2024-04-23T18:28:08.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.814+0000 7f8f2d1fe700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8f14039890 0x7f8f1403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8f18011490 tx=0x7f8f18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:08.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:08.819+0000 7f8f1effd700 1 -- 172.21.15.132:0/1549788905 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f24013070 con 0x7f8f28101990 2024-04-23T18:28:09.004 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:08 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:08.674+0000 7fed7f4c5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:28:09.005 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:08 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:08.842+0000 7fed7f4c5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:28:09.005 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:09.011+0000 7fed7f4c5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:28:09.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:09.044+0000 7f8f2fc63700 1 -- 172.21.15.132:0/1549788905 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8f28062e80 con 0x7f8f14039890 2024-04-23T18:28:09.190 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:09.190 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.190 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.190 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.190 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.190 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:09.190 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:09.190 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.190 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:28:09.191 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:28:09.191 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:09.191 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:28:09.191 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:28:09.191 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:09.191 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:09.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:09.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:09.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:28:09.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:28:09.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:09.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:28:09.266 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:28:09.266 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:09.266 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.290 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:09.010+0000 7f8be5bfb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:28:09.290 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:09.102+0000 7f8be5bfb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:28:09.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:09.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:09.579 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:09.295+0000 7fed7f4c5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:28:09.580 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:09.385+0000 7fed7f4c5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:28:09.580 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:09.586+0000 7fed7f4c5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:28:09.679 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:09.344+0000 7f8be5bfb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:28:09.679 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:09.476+0000 7f8be5bfb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:28:09.679 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:09.570+0000 7f8be5bfb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:28:09.679 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:09.685+0000 7f8be5bfb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:28:10.014 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:09 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:09.775+0000 7fed7f4c5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:28:10.290 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:10 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:10.004+0000 7f8be5bfb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:28:10.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:10 smithi157 ceph-mon[32580]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T18:28:10.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:10 smithi157 ceph-mon[32580]: Reconfiguring daemon mgr.x on smithi132 2024-04-23T18:28:10.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:10 smithi157 ceph-mon[32580]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:28:10.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:10.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:10.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:10.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:10.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:10.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:10.312 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:10 smithi132 ceph-mon[29178]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T18:28:10.312 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:10 smithi132 ceph-mon[29178]: Reconfiguring daemon mgr.x on smithi132 2024-04-23T18:28:10.312 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:10 smithi132 ceph-mon[29178]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:28:10.312 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:10.312 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:10.312 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:10.312 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:10.312 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:10.312 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:10.514 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:10 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:10.374+0000 7fed7f4c5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:28:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:10 smithi148 ceph-mon[32311]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T18:28:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:10 smithi148 ceph-mon[32311]: Reconfiguring daemon mgr.x on smithi132 2024-04-23T18:28:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:10 smithi148 ceph-mon[32311]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:28:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:11.040 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:10 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:10.604+0000 7f8be5bfb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:28:11.040 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:28:10 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:28:10.816+0000 7f8be5bfb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:28:11.082 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:10 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:10.792+0000 7fed7f4c5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:28:11.082 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:10 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:10.889+0000 7fed7f4c5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:28:11.082 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:11 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:11.089+0000 7fed7f4c5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:28:11.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:11.141+0000 7f8f1effd700 1 -- 172.21.15.132:0/1549788905 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 14) ==== 92603+0+0 (secure 0 0 0) 0x7f8f2404ed10 con 0x7f8f28101990 2024-04-23T18:28:11.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:11 smithi132 ceph-mon[29178]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:11.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:11 smithi132 ceph-mon[29178]: Standby manager daemon z started 2024-04-23T18:28:11.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:11 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.157:0/116482146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:28:11.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:11 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.157:0/116482146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:28:11.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:11 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.157:0/116482146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:28:11.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:11 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.157:0/116482146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:28:11.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:11 smithi148 ceph-mon[32311]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:11.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:11 smithi148 ceph-mon[32311]: Standby manager daemon z started 2024-04-23T18:28:11.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:11 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.157:0/116482146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:28:11.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:11 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.157:0/116482146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:28:11.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:11 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.157:0/116482146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:28:11.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:11 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.157:0/116482146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:28:11.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:11 smithi157 ceph-mon[32580]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:11.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:11 smithi157 ceph-mon[32580]: Standby manager daemon z started 2024-04-23T18:28:11.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:11 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.157:0/116482146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:28:11.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:11 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.157:0/116482146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:28:11.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:11 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.157:0/116482146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:28:11.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:11 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.157:0/116482146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:28:11.959 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:11 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:11.637+0000 7fed7f4c5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:28:11.959 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:11 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:11.726+0000 7fed7f4c5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:28:12.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:12 smithi148 ceph-mon[32311]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-23T18:28:12.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:28:12.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:12 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1500508817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5387e5b6-e309-4bf9-abcd-bd917b437059"}]: dispatch 2024-04-23T18:28:12.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:12 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1500508817' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5387e5b6-e309-4bf9-abcd-bd917b437059"}]': finished 2024-04-23T18:28:12.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:12 smithi148 ceph-mon[32311]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T18:28:12.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:12.265 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:11 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:11.965+0000 7fed7f4c5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:28:12.265 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:12 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:12.097+0000 7fed7f4c5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:28:12.265 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:12 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:12.191+0000 7fed7f4c5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:28:12.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:12 smithi157 ceph-mon[32580]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-23T18:28:12.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:28:12.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:12 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1500508817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5387e5b6-e309-4bf9-abcd-bd917b437059"}]: dispatch 2024-04-23T18:28:12.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:12 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1500508817' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5387e5b6-e309-4bf9-abcd-bd917b437059"}]': finished 2024-04-23T18:28:12.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:12 smithi157 ceph-mon[32580]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T18:28:12.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:12.602 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:12 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:12.306+0000 7fed7f4c5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:28:12.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:12 smithi132 ceph-mon[29178]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-23T18:28:12.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:28:12.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:12 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1500508817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5387e5b6-e309-4bf9-abcd-bd917b437059"}]: dispatch 2024-04-23T18:28:12.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:12 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1500508817' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5387e5b6-e309-4bf9-abcd-bd917b437059"}]': finished 2024-04-23T18:28:12.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:12 smithi132 ceph-mon[29178]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T18:28:12.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:13.015 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:12 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:12.609+0000 7fed7f4c5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:28:13.413 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:13 smithi148 ceph-mon[32311]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:13.414 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:13 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/4255557937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:28:13.414 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:13 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:13.207+0000 7fed7f4c5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:28:13.414 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:28:13 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:28:13.420+0000 7fed7f4c5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:28:13.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:13 smithi157 ceph-mon[32580]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:13.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:13 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/4255557937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:28:13.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:13 smithi132 ceph-mon[29178]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:13.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:13 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/4255557937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:28:14.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:14.152+0000 7f8f1effd700 1 -- 172.21.15.132:0/1549788905 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f8f24017440 con 0x7f8f28101990 2024-04-23T18:28:14.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:14 smithi148 ceph-mon[32311]: Standby manager daemon y started 2024-04-23T18:28:14.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:14 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.148:0/370594484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:28:14.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:14 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.148:0/370594484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:28:14.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:14 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.148:0/370594484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:28:14.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:14 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.148:0/370594484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:28:14.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:14 smithi157 ceph-mon[32580]: Standby manager daemon y started 2024-04-23T18:28:14.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:14 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.148:0/370594484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:28:14.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:14 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.148:0/370594484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:28:14.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:14 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.148:0/370594484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:28:14.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:14 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.148:0/370594484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:28:14.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:14 smithi132 ceph-mon[29178]: Standby manager daemon y started 2024-04-23T18:28:14.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:14 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.148:0/370594484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:28:14.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:14 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.148:0/370594484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:28:14.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:14 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.148:0/370594484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:28:14.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:14 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.148:0/370594484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:28:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:15 smithi148 ceph-mon[32311]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:15 smithi148 ceph-mon[32311]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-23T18:28:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:15 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:28:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:15 smithi157 ceph-mon[32580]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:15 smithi157 ceph-mon[32580]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-23T18:28:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:15 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:28:15.583 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:15 smithi132 ceph-mon[29178]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:15.583 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:15 smithi132 ceph-mon[29178]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-23T18:28:15.583 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:15 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:28:17.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:17 smithi132 ceph-mon[29178]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:17.434 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:17 smithi148 ceph-mon[32311]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:17.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:17 smithi157 ceph-mon[32580]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:19.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:19 smithi132 ceph-mon[29178]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:19.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:19 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T18:28:19.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:19 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:19.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:19 smithi148 ceph-mon[32311]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:19.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:19 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T18:28:19.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:19 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:19.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:19 smithi157 ceph-mon[32580]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:19.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:19 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T18:28:19.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:19 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:20.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:20 smithi132 ceph-mon[29178]: Deploying daemon osd.0 on smithi132 2024-04-23T18:28:20.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:20 smithi148 ceph-mon[32311]: Deploying daemon osd.0 on smithi132 2024-04-23T18:28:20.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:20 smithi157 ceph-mon[32580]: Deploying daemon osd.0 on smithi132 2024-04-23T18:28:21.347 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:21 smithi132 ceph-mon[29178]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:21.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:21 smithi148 ceph-mon[32311]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:21.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:21 smithi157 ceph-mon[32580]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:23.313 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:23 smithi132 ceph-mon[29178]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:23.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:23 smithi148 ceph-mon[32311]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:23.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:23 smithi157 ceph-mon[32580]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:25.308 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:25 smithi132 ceph-mon[29178]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:25.308 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:25 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:25.308 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:25 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:25.308 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:25 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:25 smithi148 ceph-mon[32311]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:25 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:25 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:25 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:25 smithi157 ceph-mon[32580]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:25 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:25 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:25 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:27.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:27 smithi148 ceph-mon[32311]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:27.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:27 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:27.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:27 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:27.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:27 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:27.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:27 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:27.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:27 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:27.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:27 smithi157 ceph-mon[32580]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:27.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:27 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:27.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:27 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:27.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:27 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:27.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:27 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:27.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:27 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:27.541 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:27 smithi132 ceph-mon[29178]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:27.541 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:27 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:27.541 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:27 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:27.541 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:27 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:27.541 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:27 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:27.541 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:27 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:27.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.922+0000 7f8f1effd700 1 -- 172.21.15.132:0/1549788905 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8f28062e80 con 0x7f8f14039890 2024-04-23T18:28:27.916 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 0 on host 'smithi132' 2024-04-23T18:28:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.924+0000 7f8f2fc63700 1 -- 172.21.15.132:0/1549788905 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8f14039890 msgr2=0x7f8f1403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:27.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.924+0000 7f8f2fc63700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8f14039890 0x7f8f1403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8f18011490 tx=0x7f8f18009260 comp rx=0 tx=0).stop 2024-04-23T18:28:27.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.924+0000 7f8f2fc63700 1 -- 172.21.15.132:0/1549788905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f28101990 msgr2=0x7f8f2806bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:27.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.924+0000 7f8f2fc63700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f28101990 0x7f8f2806bac0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f240036f0 tx=0x7f8f2400f710 comp rx=0 tx=0).stop 2024-04-23T18:28:27.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.925+0000 7f8f2fc63700 1 -- 172.21.15.132:0/1549788905 shutdown_connections 2024-04-23T18:28:27.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.925+0000 7f8f2fc63700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28103530 0x7f8f281a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:27.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.925+0000 7f8f2fc63700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8f14039890 0x7f8f1403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:27.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.925+0000 7f8f2fc63700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f28101990 0x7f8f2806bac0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:27.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.925+0000 7f8f2fc63700 1 --2- 172.21.15.132:0/1549788905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f28102b90 0x7f8f2819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:27.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.925+0000 7f8f2fc63700 1 -- 172.21.15.132:0/1549788905 >> 172.21.15.132:0/1549788905 conn(0x7f8f280fcf30 msgr2=0x7f8f28105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:27.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.925+0000 7f8f2fc63700 1 -- 172.21.15.132:0/1549788905 shutdown_connections 2024-04-23T18:28:27.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:28:27.925+0000 7f8f2fc63700 1 -- 172.21.15.132:0/1549788905 wait complete. 2024-04-23T18:28:28.614 DEBUG:teuthology.orchestra.run.smithi132:osd.0> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.0.service 2024-04-23T18:28:28.616 INFO:tasks.cephadm:Deploying osd.1 on smithi148 with /dev/vg_nvme/lv_4... 2024-04-23T18:28:28.616 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T18:28:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:28:28.984 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:28.984 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:28.984 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:28.984 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:28 smithi148 ceph-mon[32311]: from='osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T18:28:28.984 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:28 smithi148 ceph-mon[32311]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:29.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:29.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:29.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:29.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:28 smithi132 ceph-mon[29178]: from='osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T18:28:29.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:28 smithi132 ceph-mon[29178]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:29.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:29.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:29.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:29.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:28 smithi157 ceph-mon[32580]: from='osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T18:28:29.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:28 smithi157 ceph-mon[32580]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:30.103 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:29 smithi132 ceph-mon[29178]: from='osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T18:28:30.103 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:29 smithi132 ceph-mon[29178]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T18:28:30.103 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:29 smithi132 ceph-mon[29178]: from='osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-04-23T18:28:30.103 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:29 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:30.103 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:28:29 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0[43945]: 2024-04-23T18:28:29.903+0000 7f2bcd0a2700 -1 osd.0 0 waiting for initial osdmap 2024-04-23T18:28:30.104 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:28:29 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0[43945]: 2024-04-23T18:28:29.906+0000 7f2bc8e95700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:28:30.196 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:29 smithi148 ceph-mon[32311]: from='osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T18:28:30.197 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:29 smithi148 ceph-mon[32311]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T18:28:30.197 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:29 smithi148 ceph-mon[32311]: from='osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-04-23T18:28:30.197 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:29 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:30.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:29 smithi157 ceph-mon[32580]: from='osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T18:28:30.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:29 smithi157 ceph-mon[32580]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T18:28:30.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:29 smithi157 ceph-mon[32580]: from='osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-04-23T18:28:30.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:29 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:30.761 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T18:28:30.762 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T18:28:30.762 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-04-23T18:28:30.762 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-04-23T18:28:30.762 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129716 s, 808 MB/s 2024-04-23T18:28:30.762 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-04-23T18:28:31.085 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:30 smithi148 ceph-mon[32311]: from='osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-04-23T18:28:31.085 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:30 smithi148 ceph-mon[32311]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T18:28:31.085 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:30 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:31.085 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:30 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:31.085 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:30 smithi148 ceph-mon[32311]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:31.219 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:30 smithi132 ceph-mon[29178]: from='osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-04-23T18:28:31.220 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:30 smithi132 ceph-mon[29178]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T18:28:31.220 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:30 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:31.220 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:30 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:31.220 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:30 smithi132 ceph-mon[29178]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:31.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:30 smithi157 ceph-mon[32580]: from='osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-04-23T18:28:31.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:30 smithi157 ceph-mon[32580]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T18:28:31.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:30 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:31.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:30 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:31.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:30 smithi157 ceph-mon[32580]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:28:31.373 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_4 2024-04-23T18:28:31.620 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:28:31.972 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: purged_snaps scrub starts 2024-04-23T18:28:31.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: purged_snaps scrub ok 2024-04-23T18:28:31.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:31.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150] boot 2024-04-23T18:28:31.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T18:28:31.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:31.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:31.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:31.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:28:31.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:31.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:31.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:31.974 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:32.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: purged_snaps scrub starts 2024-04-23T18:28:32.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: purged_snaps scrub ok 2024-04-23T18:28:32.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:32.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150] boot 2024-04-23T18:28:32.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T18:28:32.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:32.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:32.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:32.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:28:32.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:32.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:32.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:32.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:32.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: purged_snaps scrub starts 2024-04-23T18:28:32.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: purged_snaps scrub ok 2024-04-23T18:28:32.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:32.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: osd.0 [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150] boot 2024-04-23T18:28:32.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T18:28:32.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:28:32.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:32.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:32.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:28:32.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:32.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:32.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:32.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:33.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.116+0000 7fbf53b01700 1 -- 172.21.15.148:0/2878582136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf4c101990 msgr2=0x7fbf4c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:33.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.116+0000 7fbf53b01700 1 --2- 172.21.15.148:0/2878582136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf4c101990 0x7fbf4c101db0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf480099c0 tx=0x7fbf48009cd0 comp rx=0 tx=0).stop 2024-04-23T18:28:33.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.116+0000 7fbf53b01700 1 -- 172.21.15.148:0/2878582136 shutdown_connections 2024-04-23T18:28:33.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.116+0000 7fbf53b01700 1 --2- 172.21.15.148:0/2878582136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c103530 0x7fbf4c109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:33.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.116+0000 7fbf53b01700 1 --2- 172.21.15.148:0/2878582136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf4c101990 0x7fbf4c101db0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:33.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.116+0000 7fbf53b01700 1 --2- 172.21.15.148:0/2878582136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf4c102b90 0x7fbf4c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:33.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.116+0000 7fbf53b01700 1 -- 172.21.15.148:0/2878582136 >> 172.21.15.148:0/2878582136 conn(0x7fbf4c0fcf30 msgr2=0x7fbf4c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:33.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.116+0000 7fbf53b01700 1 -- 172.21.15.148:0/2878582136 shutdown_connections 2024-04-23T18:28:33.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.116+0000 7fbf53b01700 1 -- 172.21.15.148:0/2878582136 wait complete. 2024-04-23T18:28:33.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf53b01700 1 Processor -- start 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf53b01700 1 -- start start 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf53b01700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf4c101990 0x7fbf4c105920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf53b01700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf4c102b90 0x7fbf4c103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf53b01700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c103530 0x7fbf4c1044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf53b01700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c10d480 con 0x7fbf4c102b90 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf53b01700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c10d300 con 0x7fbf4c101990 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf53b01700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c10d600 con 0x7fbf4c103530 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf5189d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf4c101990 0x7fbf4c105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf5189d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf4c101990 0x7fbf4c105920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55938/0 (socket says 172.21.15.148:55938) 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf5189d700 1 -- 172.21.15.148:0/6586605 learned_addr learned my addr 172.21.15.148:0/6586605 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf5109c700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf4c102b90 0x7fbf4c103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf5189d700 1 -- 172.21.15.148:0/6586605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c103530 msgr2=0x7fbf4c1044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:33.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf5209e700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c103530 0x7fbf4c1044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:33.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf5189d700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c103530 0x7fbf4c1044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:33.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf5189d700 1 -- 172.21.15.148:0/6586605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf4c102b90 msgr2=0x7fbf4c103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:33.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf5189d700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf4c102b90 0x7fbf4c103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:33.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.117+0000 7fbf5189d700 1 -- 172.21.15.148:0/6586605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4c104d60 con 0x7fbf4c101990 2024-04-23T18:28:33.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.118+0000 7fbf5209e700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c103530 0x7fbf4c1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:28:33.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.118+0000 7fbf5189d700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf4c101990 0x7fbf4c105920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009f50 tx=0x7fbf480047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:33.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.118+0000 7fbf42ffd700 1 -- 172.21.15.148:0/6586605 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48027070 con 0x7fbf4c101990 2024-04-23T18:28:33.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.118+0000 7fbf53b01700 1 -- 172.21.15.148:0/6586605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4c1a2cd0 con 0x7fbf4c101990 2024-04-23T18:28:33.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.118+0000 7fbf42ffd700 1 -- 172.21.15.148:0/6586605 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf48022470 con 0x7fbf4c101990 2024-04-23T18:28:33.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.118+0000 7fbf42ffd700 1 -- 172.21.15.148:0/6586605 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4800f6c0 con 0x7fbf4c101990 2024-04-23T18:28:33.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.118+0000 7fbf53b01700 1 -- 172.21.15.148:0/6586605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf4c1a3180 con 0x7fbf4c101990 2024-04-23T18:28:33.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.119+0000 7fbf53b01700 1 -- 172.21.15.148:0/6586605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf4c069730 con 0x7fbf4c101990 2024-04-23T18:28:33.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.120+0000 7fbf42ffd700 1 -- 172.21.15.148:0/6586605 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7fbf4802a070 con 0x7fbf4c101990 2024-04-23T18:28:33.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.120+0000 7fbf42ffd700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fbf380a3bb0 0x7fbf380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:33.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.121+0000 7fbf42ffd700 1 -- 172.21.15.148:0/6586605 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fbf480cf8e0 con 0x7fbf4c101990 2024-04-23T18:28:33.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.121+0000 7fbf5109c700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fbf380a3bb0 0x7fbf380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:33.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.121+0000 7fbf5109c700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fbf380a3bb0 0x7fbf380a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00ac50 tx=0x7fbf3c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:33.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.125+0000 7fbf42ffd700 1 -- 172.21.15.148:0/6586605 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf48013070 con 0x7fbf4c101990 2024-04-23T18:28:33.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:32 smithi157 ceph-mon[32580]: Detected new or changed devices on smithi132 2024-04-23T18:28:33.192 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:32 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi132 to 17088M 2024-04-23T18:28:33.192 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:32 smithi157 ceph-mon[32580]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T18:28:33.192 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:32 smithi157 ceph-mon[32580]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:33.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:32 smithi148 ceph-mon[32311]: Detected new or changed devices on smithi132 2024-04-23T18:28:33.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:32 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi132 to 17088M 2024-04-23T18:28:33.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:32 smithi148 ceph-mon[32311]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T18:28:33.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:32 smithi148 ceph-mon[32311]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:33.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:33.332+0000 7fbf53b01700 1 -- 172.21.15.148:0/6586605 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbf4c0ff120 con 0x7fbf380a3bb0 2024-04-23T18:28:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:32 smithi132 ceph-mon[29178]: Detected new or changed devices on smithi132 2024-04-23T18:28:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:32 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi132 to 17088M 2024-04-23T18:28:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:32 smithi132 ceph-mon[29178]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T18:28:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:32 smithi132 ceph-mon[29178]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:34.231 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:33 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:28:34.231 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:33 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:28:34.232 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:33 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:34.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:33 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:28:34.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:33 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:28:34.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:33 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:34.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:33 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:28:34.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:33 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:28:34.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:33 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:35.150 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:34 smithi148 ceph-mon[32311]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:28:35.150 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:34 smithi148 ceph-mon[32311]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:35.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:34 smithi157 ceph-mon[32580]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:28:35.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:34 smithi157 ceph-mon[32580]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:34 smithi132 ceph-mon[29178]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:28:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:34 smithi132 ceph-mon[29178]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:36.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:36 smithi148 ceph-mon[32311]: from='client.? 172.21.15.148:0/2862438329' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "962fbcd8-143f-4f15-9e83-6afed0c5a72d"}]: dispatch 2024-04-23T18:28:36.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:36 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "962fbcd8-143f-4f15-9e83-6afed0c5a72d"}]: dispatch 2024-04-23T18:28:36.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:36 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "962fbcd8-143f-4f15-9e83-6afed0c5a72d"}]': finished 2024-04-23T18:28:36.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:36 smithi148 ceph-mon[32311]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T18:28:36.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:36 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:36.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:36 smithi132 ceph-mon[29178]: from='client.? 172.21.15.148:0/2862438329' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "962fbcd8-143f-4f15-9e83-6afed0c5a72d"}]: dispatch 2024-04-23T18:28:36.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:36 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "962fbcd8-143f-4f15-9e83-6afed0c5a72d"}]: dispatch 2024-04-23T18:28:36.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:36 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "962fbcd8-143f-4f15-9e83-6afed0c5a72d"}]': finished 2024-04-23T18:28:36.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:36 smithi132 ceph-mon[29178]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T18:28:36.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:36 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:36.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:36 smithi157 ceph-mon[32580]: from='client.? 172.21.15.148:0/2862438329' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "962fbcd8-143f-4f15-9e83-6afed0c5a72d"}]: dispatch 2024-04-23T18:28:36.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:36 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "962fbcd8-143f-4f15-9e83-6afed0c5a72d"}]: dispatch 2024-04-23T18:28:36.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:36 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "962fbcd8-143f-4f15-9e83-6afed0c5a72d"}]': finished 2024-04-23T18:28:36.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:36 smithi157 ceph-mon[32580]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T18:28:36.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:36 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:37.349 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:37 smithi132 ceph-mon[29178]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:37.349 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:37 smithi132 ceph-mon[29178]: from='client.? 172.21.15.148:0/116708681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:28:37.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:37 smithi148 ceph-mon[32311]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:37.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:37 smithi148 ceph-mon[32311]: from='client.? 172.21.15.148:0/116708681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:28:37.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:37 smithi157 ceph-mon[32580]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:37.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:37 smithi157 ceph-mon[32580]: from='client.? 172.21.15.148:0/116708681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:28:39.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:39 smithi132 ceph-mon[29178]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:39.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:39 smithi148 ceph-mon[32311]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:39.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:39 smithi157 ceph-mon[32580]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:41.220 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:41 smithi148 ceph-mon[32311]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:41.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:41 smithi132 ceph-mon[29178]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:41 smithi157 ceph-mon[32580]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:43.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:43 smithi148 ceph-mon[32311]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:43.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:43 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T18:28:43.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:43 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:43.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:43 smithi157 ceph-mon[32580]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:43.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:43 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T18:28:43.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:43 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:43.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:43 smithi132 ceph-mon[29178]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:43.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:43 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T18:28:43.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:43 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:44.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:44 smithi157 ceph-mon[32580]: Deploying daemon osd.1 on smithi148 2024-04-23T18:28:44.409 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:44 smithi148 ceph-mon[32311]: Deploying daemon osd.1 on smithi148 2024-04-23T18:28:44.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:44 smithi132 ceph-mon[29178]: Deploying daemon osd.1 on smithi148 2024-04-23T18:28:45.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:45 smithi148 ceph-mon[32311]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:45.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:45 smithi157 ceph-mon[32580]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:45.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:45 smithi132 ceph-mon[29178]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:47.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:47 smithi148 ceph-mon[32311]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:47.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:47 smithi157 ceph-mon[32580]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:47.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:47 smithi132 ceph-mon[29178]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:49.250 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:49 smithi148 ceph-mon[32311]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:49.250 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:49.250 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:49.250 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:49 smithi157 ceph-mon[32580]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:49.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:49.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:49 smithi132 ceph-mon[29178]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:49.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:49.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:49.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:51.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:51 smithi148 ceph-mon[32311]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:51.417 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:51 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:51.417 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:51 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:51.417 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:51 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:51.417 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:51 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:51.417 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:51 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:51.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:51 smithi157 ceph-mon[32580]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:51.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:51 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:51.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:51 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:51.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:51 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:51.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:51 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:51.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:51 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:51.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:51 smithi132 ceph-mon[29178]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:51.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:51 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:51.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:51 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:51.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:51 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:51.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:51 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:51.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:51 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:52.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.095+0000 7fbf42ffd700 1 -- 172.21.15.148:0/6586605 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbf4c0ff120 con 0x7fbf380a3bb0 2024-04-23T18:28:52.089 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 1 on host 'smithi148' 2024-04-23T18:28:52.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.098+0000 7fbf53b01700 1 -- 172.21.15.148:0/6586605 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fbf380a3bb0 msgr2=0x7fbf380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:52.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.098+0000 7fbf53b01700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fbf380a3bb0 0x7fbf380a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00ac50 tx=0x7fbf3c00a380 comp rx=0 tx=0).stop 2024-04-23T18:28:52.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.098+0000 7fbf53b01700 1 -- 172.21.15.148:0/6586605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf4c101990 msgr2=0x7fbf4c105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:52.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.098+0000 7fbf53b01700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf4c101990 0x7fbf4c105920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009f50 tx=0x7fbf480047b0 comp rx=0 tx=0).stop 2024-04-23T18:28:52.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.098+0000 7fbf53b01700 1 -- 172.21.15.148:0/6586605 shutdown_connections 2024-04-23T18:28:52.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.098+0000 7fbf53b01700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c103530 0x7fbf4c1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:52.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.098+0000 7fbf53b01700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf4c101990 0x7fbf4c105920 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:52.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.098+0000 7fbf53b01700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fbf380a3bb0 0x7fbf380a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:52.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.098+0000 7fbf53b01700 1 --2- 172.21.15.148:0/6586605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf4c102b90 0x7fbf4c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:52.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.098+0000 7fbf53b01700 1 -- 172.21.15.148:0/6586605 >> 172.21.15.148:0/6586605 conn(0x7fbf4c0fcf30 msgr2=0x7fbf4c0fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:52.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.098+0000 7fbf53b01700 1 -- 172.21.15.148:0/6586605 shutdown_connections 2024-04-23T18:28:52.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:28:52.098+0000 7fbf53b01700 1 -- 172.21.15.148:0/6586605 wait complete. 2024-04-23T18:28:52.748 DEBUG:teuthology.orchestra.run.smithi148:osd.1> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.1.service 2024-04-23T18:28:52.751 INFO:tasks.cephadm:Deploying osd.2 on smithi157 with /dev/vg_nvme/lv_4... 2024-04-23T18:28:52.752 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T18:28:52.997 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:28:53.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:53 smithi148 ceph-mon[32311]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:53.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:53.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:53.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:53.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:53 smithi148 ceph-mon[32311]: from='osd.1 [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:28:53.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:53 smithi148 ceph-mon[32311]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:28:53.361 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:53 smithi157 ceph-mon[32580]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:53.362 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:53.362 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:53.362 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:53.362 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:53 smithi157 ceph-mon[32580]: from='osd.1 [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:28:53.362 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:53 smithi157 ceph-mon[32580]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:28:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:53 smithi132 ceph-mon[29178]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:53.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:53.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:53.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:53 smithi132 ceph-mon[29178]: from='osd.1 [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:28:53.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:53 smithi132 ceph-mon[29178]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:28:54.375 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:54 smithi157 ceph-mon[32580]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T18:28:54.375 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:54 smithi157 ceph-mon[32580]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T18:28:54.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:54.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:54 smithi157 ceph-mon[32580]: from='osd.1 [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:28:54.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:54 smithi157 ceph-mon[32580]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:28:54.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:54 smithi148 ceph-mon[32311]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T18:28:54.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:54 smithi148 ceph-mon[32311]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T18:28:54.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:54.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:54 smithi148 ceph-mon[32311]: from='osd.1 [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:28:54.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:54 smithi148 ceph-mon[32311]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:28:54.515 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:28:54 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1[36913]: 2024-04-23T18:28:54.160+0000 7ffa7acf5700 -1 osd.1 0 waiting for initial osdmap 2024-04-23T18:28:54.516 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:28:54 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1[36913]: 2024-04-23T18:28:54.164+0000 7ffa76ae8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:28:54.535 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:54 smithi132 ceph-mon[29178]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T18:28:54.535 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:54 smithi132 ceph-mon[29178]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T18:28:54.535 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:54.535 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:54 smithi132 ceph-mon[29178]: from='osd.1 [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:28:54.535 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:54 smithi132 ceph-mon[29178]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:28:54.948 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T18:28:54.948 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T18:28:54.948 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-04-23T18:28:54.948 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-04-23T18:28:54.948 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00841491 s, 1.2 GB/s 2024-04-23T18:28:54.948 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-04-23T18:28:55.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:55 smithi148 ceph-mon[32311]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:55.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:55 smithi148 ceph-mon[32311]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-23T18:28:55.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:55 smithi148 ceph-mon[32311]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T18:28:55.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:55.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:55.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:55 smithi157 ceph-mon[32580]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:55.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:55 smithi157 ceph-mon[32580]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-23T18:28:55.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:55 smithi157 ceph-mon[32580]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T18:28:55.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:55.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:55.461 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:55 smithi132 ceph-mon[29178]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:28:55.461 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:55 smithi132 ceph-mon[29178]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-23T18:28:55.461 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:55 smithi132 ceph-mon[29178]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T18:28:55.461 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:55.461 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:55.548 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_4 2024-04-23T18:28:55.790 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:28:56.183 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:56 smithi157 ceph-mon[32580]: purged_snaps scrub starts 2024-04-23T18:28:56.183 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:56 smithi157 ceph-mon[32580]: purged_snaps scrub ok 2024-04-23T18:28:56.183 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:56 smithi157 ceph-mon[32580]: osd.1 [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181] boot 2024-04-23T18:28:56.183 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:56 smithi157 ceph-mon[32580]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T18:28:56.183 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:56.184 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:56.184 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:56.184 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:28:56.184 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:56.184 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:56.184 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:56.184 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:56.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:56 smithi148 ceph-mon[32311]: purged_snaps scrub starts 2024-04-23T18:28:56.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:56 smithi148 ceph-mon[32311]: purged_snaps scrub ok 2024-04-23T18:28:56.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:56 smithi148 ceph-mon[32311]: osd.1 [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181] boot 2024-04-23T18:28:56.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:56 smithi148 ceph-mon[32311]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T18:28:56.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:56.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:56.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:56.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:28:56.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:56.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:56.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:56.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:56.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:56 smithi132 ceph-mon[29178]: purged_snaps scrub starts 2024-04-23T18:28:56.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:56 smithi132 ceph-mon[29178]: purged_snaps scrub ok 2024-04-23T18:28:56.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:56 smithi132 ceph-mon[29178]: osd.1 [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181] boot 2024-04-23T18:28:56.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:56 smithi132 ceph-mon[29178]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T18:28:56.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:28:56.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:56.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:28:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:56.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:28:57.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 -- 172.21.15.157:0/2301434909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0750102aa0 msgr2=0x7f0750102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:57.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 --2- 172.21.15.157:0/2301434909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0750102aa0 0x7f0750102ec0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f074c009990 tx=0x7f074c009ca0 comp rx=0 tx=0).stop 2024-04-23T18:28:57.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 -- 172.21.15.157:0/2301434909 shutdown_connections 2024-04-23T18:28:57.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 --2- 172.21.15.157:0/2301434909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0750104640 0x7f075010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:57.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 --2- 172.21.15.157:0/2301434909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0750102aa0 0x7f0750102ec0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:57.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 --2- 172.21.15.157:0/2301434909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0750103ca0 0x7f0750104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:57.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 -- 172.21.15.157:0/2301434909 >> 172.21.15.157:0/2301434909 conn(0x7f07500fe040 msgr2=0x7f0750100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:57.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 -- 172.21.15.157:0/2301434909 shutdown_connections 2024-04-23T18:28:57.369 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 -- 172.21.15.157:0/2301434909 wait complete. 2024-04-23T18:28:57.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 Processor -- start 2024-04-23T18:28:57.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 -- start start 2024-04-23T18:28:57.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0750102aa0 0x7f0750198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:57.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0750103ca0 0x7f07501990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:57.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0750104640 0x7f07501a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:57.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f075010e530 con 0x7f0750102aa0 2024-04-23T18:28:57.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f075010e3b0 con 0x7f0750104640 2024-04-23T18:28:57.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.373+0000 7f07575b3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f075010e6b0 con 0x7f0750103ca0 2024-04-23T18:28:57.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f0754b4e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0750103ca0 0x7f07501990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:57.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f0754b4e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0750103ca0 0x7f07501990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55830/0 (socket says 172.21.15.157:55830) 2024-04-23T18:28:57.370 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f0754b4e700 1 -- 172.21.15.157:0/3514581884 learned_addr learned my addr 172.21.15.157:0/3514581884 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f0755b50700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0750104640 0x7f07501a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f075534f700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0750102aa0 0x7f0750198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f0754b4e700 1 -- 172.21.15.157:0/3514581884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0750104640 msgr2=0x7f07501a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f0754b4e700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0750104640 0x7f07501a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f0754b4e700 1 -- 172.21.15.157:0/3514581884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0750102aa0 msgr2=0x7f0750198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f0754b4e700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0750102aa0 0x7f0750198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f0754b4e700 1 -- 172.21.15.157:0/3514581884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07501a07d0 con 0x7f0750103ca0 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f0754b4e700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0750103ca0 0x7f07501990b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f074000c9e0 tx=0x7f074000ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f07467fc700 1 -- 172.21.15.157:0/3514581884 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740007ac0 con 0x7f0750103ca0 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f07467fc700 1 -- 172.21.15.157:0/3514581884 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0740007c60 con 0x7f0750103ca0 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f07575b3700 1 -- 172.21.15.157:0/3514581884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07501a0ac0 con 0x7f0750103ca0 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f07467fc700 1 -- 172.21.15.157:0/3514581884 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740018660 con 0x7f0750103ca0 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.374+0000 7f07575b3700 1 -- 172.21.15.157:0/3514581884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07501a1000 con 0x7f0750103ca0 2024-04-23T18:28:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.375+0000 7f07575b3700 1 -- 172.21.15.157:0/3514581884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0750069730 con 0x7f0750103ca0 2024-04-23T18:28:57.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.376+0000 7f07467fc700 1 -- 172.21.15.157:0/3514581884 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f074000f450 con 0x7f0750103ca0 2024-04-23T18:28:57.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.377+0000 7f07467fc700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f073c0a3bb0 0x7f073c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:57.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.377+0000 7f07467fc700 1 -- 172.21.15.157:0/3514581884 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f07400ceb20 con 0x7f0750103ca0 2024-04-23T18:28:57.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.377+0000 7f075534f700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f073c0a3bb0 0x7f073c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:57.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.377+0000 7f075534f700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f073c0a3bb0 0x7f073c0a6070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f074c0038a0 tx=0x7f074c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:57.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.382+0000 7f07467fc700 1 -- 172.21.15.157:0/3514581884 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0740014030 con 0x7f0750103ca0 2024-04-23T18:28:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:57 smithi148 ceph-mon[32311]: Detected new or changed devices on smithi148 2024-04-23T18:28:57.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:57 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi148 to 17088M 2024-04-23T18:28:57.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:57 smithi148 ceph-mon[32311]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:28:57.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:57 smithi148 ceph-mon[32311]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T18:28:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:57 smithi157 ceph-mon[32580]: Detected new or changed devices on smithi148 2024-04-23T18:28:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:57 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi148 to 17088M 2024-04-23T18:28:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:57 smithi157 ceph-mon[32580]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:28:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:57 smithi157 ceph-mon[32580]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T18:28:57.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:28:57.586+0000 7f07575b3700 1 -- 172.21.15.157:0/3514581884 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0750062e80 con 0x7f073c0a3bb0 2024-04-23T18:28:57.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:57 smithi132 ceph-mon[29178]: Detected new or changed devices on smithi148 2024-04-23T18:28:57.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:57 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi148 to 17088M 2024-04-23T18:28:57.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:57 smithi132 ceph-mon[29178]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:28:57.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:57 smithi132 ceph-mon[29178]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T18:28:58.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:28:58.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:28:58.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:58.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:28:58.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:28:58.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:58.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:28:58.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:28:58.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:59.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:59 smithi157 ceph-mon[32580]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:28:59.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:28:59 smithi157 ceph-mon[32580]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:28:59.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:59 smithi148 ceph-mon[32311]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:28:59.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:28:59 smithi148 ceph-mon[32311]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:28:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:59 smithi132 ceph-mon[29178]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:28:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:28:59 smithi132 ceph-mon[29178]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:01.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:01 smithi148 ceph-mon[32311]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:01.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:01 smithi148 ceph-mon[32311]: from='client.? 172.21.15.157:0/809040760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c4885a5-c35c-4b55-bbd2-9370fc797096"}]: dispatch 2024-04-23T18:29:01.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:01 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c4885a5-c35c-4b55-bbd2-9370fc797096"}]: dispatch 2024-04-23T18:29:01.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:01 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c4885a5-c35c-4b55-bbd2-9370fc797096"}]': finished 2024-04-23T18:29:01.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:01 smithi148 ceph-mon[32311]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T18:29:01.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:01.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:01 smithi148 ceph-mon[32311]: from='client.? 172.21.15.157:0/894210407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:29:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:01 smithi157 ceph-mon[32580]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:01.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:01 smithi157 ceph-mon[32580]: from='client.? 172.21.15.157:0/809040760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c4885a5-c35c-4b55-bbd2-9370fc797096"}]: dispatch 2024-04-23T18:29:01.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:01 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c4885a5-c35c-4b55-bbd2-9370fc797096"}]: dispatch 2024-04-23T18:29:01.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:01 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c4885a5-c35c-4b55-bbd2-9370fc797096"}]': finished 2024-04-23T18:29:01.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:01 smithi157 ceph-mon[32580]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T18:29:01.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:01.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:01 smithi157 ceph-mon[32580]: from='client.? 172.21.15.157:0/894210407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:29:01.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:01 smithi132 ceph-mon[29178]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:01.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:01 smithi132 ceph-mon[29178]: from='client.? 172.21.15.157:0/809040760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c4885a5-c35c-4b55-bbd2-9370fc797096"}]: dispatch 2024-04-23T18:29:01.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:01 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c4885a5-c35c-4b55-bbd2-9370fc797096"}]: dispatch 2024-04-23T18:29:01.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:01 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c4885a5-c35c-4b55-bbd2-9370fc797096"}]': finished 2024-04-23T18:29:01.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:01 smithi132 ceph-mon[29178]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T18:29:01.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:01.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:01 smithi132 ceph-mon[29178]: from='client.? 172.21.15.157:0/894210407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:29:03.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:03 smithi148 ceph-mon[32311]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:03.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:03 smithi157 ceph-mon[32580]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:03.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:03 smithi132 ceph-mon[29178]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:05.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:05 smithi132 ceph-mon[29178]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:05.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:05 smithi157 ceph-mon[32580]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:05.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:05 smithi148 ceph-mon[32311]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:07.313 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:07 smithi157 ceph-mon[32580]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:07.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:07 smithi148 ceph-mon[32311]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:07.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:07 smithi132 ceph-mon[29178]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:08.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:08 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T18:29:08.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:08 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:08.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:08 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T18:29:08.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:08 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:08.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:08 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T18:29:08.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:08 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:09.450 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:09 smithi157 ceph-mon[32580]: Deploying daemon osd.2 on smithi157 2024-04-23T18:29:09.450 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:09 smithi157 ceph-mon[32580]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:09.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:09 smithi148 ceph-mon[32311]: Deploying daemon osd.2 on smithi157 2024-04-23T18:29:09.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:09 smithi148 ceph-mon[32311]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:09.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:09 smithi132 ceph-mon[29178]: Deploying daemon osd.2 on smithi157 2024-04-23T18:29:09.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:09 smithi132 ceph-mon[29178]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:11.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:11 smithi148 ceph-mon[32311]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:11.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:11 smithi157 ceph-mon[32580]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:11.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:11 smithi132 ceph-mon[29178]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:13.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:13 smithi157 ceph-mon[32580]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:13.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:13 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:13.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:13 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:13.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:13 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:13.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:13 smithi148 ceph-mon[32311]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:13.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:13 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:13.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:13 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:13.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:13 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:13.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:13 smithi132 ceph-mon[29178]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:13.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:13 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:13.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:13 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:13.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:13 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:15.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:15 smithi148 ceph-mon[32311]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:15 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:15 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:15 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:15 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:15 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:15 smithi157 ceph-mon[32580]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:15 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:15 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:15 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:15 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:15 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:15 smithi132 ceph-mon[29178]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:15 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:15 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:15 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:15 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:15 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:16.395 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.403+0000 7f07467fc700 1 -- 172.21.15.157:0/3514581884 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0750062e80 con 0x7f073c0a3bb0 2024-04-23T18:29:16.396 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 2 on host 'smithi157' 2024-04-23T18:29:16.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.405+0000 7f07575b3700 1 -- 172.21.15.157:0/3514581884 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f073c0a3bb0 msgr2=0x7f073c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:16.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.405+0000 7f07575b3700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f073c0a3bb0 0x7f073c0a6070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f074c0038a0 tx=0x7f074c005fd0 comp rx=0 tx=0).stop 2024-04-23T18:29:16.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.405+0000 7f07575b3700 1 -- 172.21.15.157:0/3514581884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0750103ca0 msgr2=0x7f07501990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:16.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.405+0000 7f07575b3700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0750103ca0 0x7f07501990b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f074000c9e0 tx=0x7f074000ccf0 comp rx=0 tx=0).stop 2024-04-23T18:29:16.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.405+0000 7f07575b3700 1 -- 172.21.15.157:0/3514581884 shutdown_connections 2024-04-23T18:29:16.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.405+0000 7f07575b3700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0750104640 0x7f07501a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:16.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.405+0000 7f07575b3700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f073c0a3bb0 0x7f073c0a6070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:16.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.405+0000 7f07575b3700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0750102aa0 0x7f0750198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:16.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.405+0000 7f07575b3700 1 --2- 172.21.15.157:0/3514581884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0750103ca0 0x7f07501990b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:16.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.405+0000 7f07575b3700 1 -- 172.21.15.157:0/3514581884 >> 172.21.15.157:0/3514581884 conn(0x7f07500fe040 msgr2=0x7f0750107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:16.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.405+0000 7f07575b3700 1 -- 172.21.15.157:0/3514581884 shutdown_connections 2024-04-23T18:29:16.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:29:16.405+0000 7f07575b3700 1 -- 172.21.15.157:0/3514581884 wait complete. 2024-04-23T18:29:17.067 DEBUG:teuthology.orchestra.run.smithi157:osd.2> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.2.service 2024-04-23T18:29:17.069 INFO:tasks.cephadm:Deploying osd.3 on smithi132 with /dev/vg_nvme/lv_3... 2024-04-23T18:29:17.070 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T18:29:17.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:17 smithi157 ceph-mon[32580]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:17.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:17 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:17.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:17 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:17.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:17 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:17.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:17 smithi157 ceph-mon[32580]: from='osd.2 [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:29:17.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:17 smithi157 ceph-mon[32580]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:29:17.316 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:29:17.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:17 smithi132 ceph-mon[29178]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:17.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:17 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:17.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:17 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:17.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:17 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:17.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:17 smithi132 ceph-mon[29178]: from='osd.2 [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:29:17.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:17 smithi132 ceph-mon[29178]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:29:17.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:17 smithi148 ceph-mon[32311]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:17.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:17 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:17.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:17 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:17.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:17 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:17.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:17 smithi148 ceph-mon[32311]: from='osd.2 [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:29:17.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:17 smithi148 ceph-mon[32311]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:29:18.471 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:18 smithi157 ceph-mon[32580]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T18:29:18.472 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:18 smithi157 ceph-mon[32580]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T18:29:18.472 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:18 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:18.472 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:18 smithi157 ceph-mon[32580]: from='osd.2 [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:29:18.472 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:18 smithi157 ceph-mon[32580]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:29:18.472 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:18 smithi157 ceph-mon[32580]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:18.543 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:18 smithi132 ceph-mon[29178]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T18:29:18.543 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:18 smithi132 ceph-mon[29178]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T18:29:18.543 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:18 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:18.543 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:18 smithi132 ceph-mon[29178]: from='osd.2 [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:29:18.543 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:18 smithi132 ceph-mon[29178]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:29:18.544 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:18 smithi132 ceph-mon[29178]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:18.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:18 smithi148 ceph-mon[32311]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T18:29:18.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:18 smithi148 ceph-mon[32311]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T18:29:18.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:18 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:18.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:18 smithi148 ceph-mon[32311]: from='osd.2 [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:29:18.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:18 smithi148 ceph-mon[32311]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:29:18.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:18 smithi148 ceph-mon[32311]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:19.261 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T18:29:19.262 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T18:29:19.262 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-04-23T18:29:19.262 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-04-23T18:29:19.262 INFO:teuthology.orchestra.run.smithi132.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128178 s, 818 MB/s 2024-04-23T18:29:19.262 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-04-23T18:29:19.290 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:29:18 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2[37705]: 2024-04-23T18:29:18.821+0000 7f4feb3f9700 -1 osd.2 0 waiting for initial osdmap 2024-04-23T18:29:19.290 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:29:18 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2[37705]: 2024-04-23T18:29:18.824+0000 7f4fe31e4700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:29:19.571 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:19 smithi132 ceph-mon[29178]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-23T18:29:19.571 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:19 smithi132 ceph-mon[29178]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T18:29:19.571 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:19 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:19.571 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:19 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:19.571 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:19 smithi132 ceph-mon[29178]: from='osd.2 ' entity='osd.2' 2024-04-23T18:29:19.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:19 smithi157 ceph-mon[32580]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-23T18:29:19.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:19 smithi157 ceph-mon[32580]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T18:29:19.643 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:19 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:19.643 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:19 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:19.643 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:19 smithi157 ceph-mon[32580]: from='osd.2 ' entity='osd.2' 2024-04-23T18:29:19.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:19 smithi148 ceph-mon[32311]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-23T18:29:19.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:19 smithi148 ceph-mon[32311]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T18:29:19.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:19 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:19.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:19 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:19.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:19 smithi148 ceph-mon[32311]: from='osd.2 ' entity='osd.2' 2024-04-23T18:29:19.973 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi132:vg_nvme/lv_3 2024-04-23T18:29:20.219 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:29:20.598 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: purged_snaps scrub starts 2024-04-23T18:29:20.598 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: purged_snaps scrub ok 2024-04-23T18:29:20.598 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:20.598 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: osd.2 [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857] boot 2024-04-23T18:29:20.598 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T18:29:20.598 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:20.598 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: Detected new or changed devices on smithi157 2024-04-23T18:29:20.598 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:20.599 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:20.599 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:29:20.599 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi157 to 17088M 2024-04-23T18:29:20.599 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:20.599 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:20.599 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:20.599 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:20.599 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:20.599 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: purged_snaps scrub starts 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: purged_snaps scrub ok 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: osd.2 [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857] boot 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: Detected new or changed devices on smithi157 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi157 to 17088M 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:20.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:20.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:20.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:20.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:29:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: purged_snaps scrub starts 2024-04-23T18:29:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: purged_snaps scrub ok 2024-04-23T18:29:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: osd.2 [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857] boot 2024-04-23T18:29:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T18:29:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:29:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: Detected new or changed devices on smithi157 2024-04-23T18:29:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:29:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi157 to 17088M 2024-04-23T18:29:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:29:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:29:21.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.882+0000 7f189b55f700 1 -- 172.21.15.132:0/2780581814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1894103540 msgr2=0x7f1894109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:21.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.882+0000 7f189b55f700 1 --2- 172.21.15.132:0/2780581814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1894103540 0x7f1894109df0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f188c00b0d0 tx=0x7f188c00b3e0 comp rx=0 tx=0).stop 2024-04-23T18:29:21.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.882+0000 7f189b55f700 1 -- 172.21.15.132:0/2780581814 shutdown_connections 2024-04-23T18:29:21.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.882+0000 7f189b55f700 1 --2- 172.21.15.132:0/2780581814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1894103540 0x7f1894109df0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:21.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.882+0000 7f189b55f700 1 --2- 172.21.15.132:0/2780581814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18941019a0 0x7f1894101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:21.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.882+0000 7f189b55f700 1 --2- 172.21.15.132:0/2780581814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894102ba0 0x7f1894103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:21.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.882+0000 7f189b55f700 1 -- 172.21.15.132:0/2780581814 >> 172.21.15.132:0/2780581814 conn(0x7f18940fcf40 msgr2=0x7f18940ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:21.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.882+0000 7f189b55f700 1 -- 172.21.15.132:0/2780581814 shutdown_connections 2024-04-23T18:29:21.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.882+0000 7f189b55f700 1 -- 172.21.15.132:0/2780581814 wait complete. 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.882+0000 7f189b55f700 1 Processor -- start 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f189b55f700 1 -- start start 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f189b55f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18941019a0 0x7f1894197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f189b55f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894102ba0 0x7f1894198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f189b55f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1894103540 0x7f189419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f189b55f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f189410d420 con 0x7f18941019a0 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f189b55f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f189410d2a0 con 0x7f1894102ba0 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f189b55f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f189410d5a0 con 0x7f1894103540 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f18992fb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18941019a0 0x7f1894197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f18992fb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18941019a0 0x7f1894197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51254/0 (socket says 172.21.15.132:51254) 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f18992fb700 1 -- 172.21.15.132:0/2508009000 learned_addr learned my addr 172.21.15.132:0/2508009000 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f1898afa700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894102ba0 0x7f1894198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:21.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f1899afc700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1894103540 0x7f189419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:21.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f18992fb700 1 -- 172.21.15.132:0/2508009000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1894103540 msgr2=0x7f189419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:21.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f18992fb700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1894103540 0x7f189419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:21.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f18992fb700 1 -- 172.21.15.132:0/2508009000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894102ba0 msgr2=0x7f1894198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:21.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f18992fb700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894102ba0 0x7f1894198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:21.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f18992fb700 1 -- 172.21.15.132:0/2508009000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189419f720 con 0x7f18941019a0 2024-04-23T18:29:21.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f18992fb700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18941019a0 0x7f1894197ac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1890007b80 tx=0x7f1890007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:21.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f188a7fc700 1 -- 172.21.15.132:0/2508009000 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890019020 con 0x7f18941019a0 2024-04-23T18:29:21.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f188a7fc700 1 -- 172.21.15.132:0/2508009000 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18900044d0 con 0x7f18941019a0 2024-04-23T18:29:21.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f189b55f700 1 -- 172.21.15.132:0/2508009000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189406a780 con 0x7f18941019a0 2024-04-23T18:29:21.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.883+0000 7f188a7fc700 1 -- 172.21.15.132:0/2508009000 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189000e540 con 0x7f18941019a0 2024-04-23T18:29:21.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.884+0000 7f189b55f700 1 -- 172.21.15.132:0/2508009000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189406acc0 con 0x7f18941019a0 2024-04-23T18:29:21.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.885+0000 7f189b55f700 1 -- 172.21.15.132:0/2508009000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1894069730 con 0x7f18941019a0 2024-04-23T18:29:21.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.886+0000 7f188a7fc700 1 -- 172.21.15.132:0/2508009000 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f18900036a0 con 0x7f18941019a0 2024-04-23T18:29:21.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.886+0000 7f188a7fc700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f18800a3bb0 0x7f18800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:21.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.886+0000 7f1898afa700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f18800a3bb0 0x7f18800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:21.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.886+0000 7f188a7fc700 1 -- 172.21.15.132:0/2508009000 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f18900d05d0 con 0x7f18941019a0 2024-04-23T18:29:21.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.887+0000 7f1898afa700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f18800a3bb0 0x7f18800a6070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1884005950 tx=0x7f188400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:21.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:21.890+0000 7f188a7fc700 1 -- 172.21.15.132:0/2508009000 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1890023020 con 0x7f18941019a0 2024-04-23T18:29:22.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:22.109+0000 7f189b55f700 1 -- 172.21.15.132:0/2508009000 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f189406af20 con 0x7f18800a3bb0 2024-04-23T18:29:22.286 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:22 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T18:29:22.287 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:22 smithi157 ceph-mon[32580]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T18:29:22.287 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:22 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:29:22.287 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41626]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:29:22.287 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41626]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:29:22.287 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41626]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:29:22.287 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41626]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:29:22.324 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:22 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T18:29:22.324 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:22 smithi132 ceph-mon[29178]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T18:29:22.324 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:22 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:29:22.325 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48376]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:29:22.325 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48376]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:29:22.325 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48376]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:29:22.325 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48376]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:29:22.325 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48411]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:29:22.325 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48411]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:29:22.325 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48411]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:29:22.325 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48411]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:29:22.397 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:22 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T18:29:22.397 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:22 smithi148 ceph-mon[32311]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T18:29:22.397 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:22 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:29:22.398 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40766]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:29:22.398 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40766]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:29:22.398 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40766]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:29:22.398 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40766]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:29:22.398 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40769]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:29:22.398 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40769]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:29:22.398 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40769]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:29:22.398 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40769]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:29:22.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41632]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:29:22.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41632]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:29:22.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41632]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:29:22.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41632]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:29:22.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41635]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:29:22.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41635]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:29:22.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41635]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:29:22.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41635]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:29:22.541 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41629]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:29:22.541 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41629]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:29:22.541 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41629]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:29:22.541 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:29:22 smithi157 sudo[41629]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:29:22.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48432]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:29:22.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48432]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:29:22.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48432]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:29:22.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48432]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:29:22.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48435]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:29:22.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48435]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:29:22.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48435]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:29:22.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:22 smithi132 sudo[48435]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:29:22.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40772]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:29:22.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40772]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:29:22.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40772]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:29:22.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40772]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:29:22.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40775]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:29:22.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40775]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:29:22.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40775]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:29:22.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:22 smithi148 sudo[40775]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:29:23.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:29:23.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:29:23.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:29:23.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:29:23.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:29:23.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:29:23.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:29:23.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:29:23.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:29:23.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:29:23.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:29:23.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:23 smithi157 ceph-mon[32580]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:29:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:29:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T18:29:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:29:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:29:23.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:29:23.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:29:23.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:29:23.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:29:23.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:29:23.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:23 smithi132 ceph-mon[29178]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:29:23.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:29:23.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T18:29:23.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:23.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:29:23.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:29:23.517 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:29:23.517 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:29:23.517 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:29:23.517 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:29:23.517 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:23 smithi148 ceph-mon[32311]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:29:24.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:24 smithi157 ceph-mon[32580]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T18:29:24.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:24 smithi132 ceph-mon[29178]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T18:29:24.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:24 smithi148 ceph-mon[32311]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T18:29:25.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:25.058+0000 7f188a7fc700 1 -- 172.21.15.132:0/2508009000 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f189009a470 con 0x7f18941019a0 2024-04-23T18:29:25.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:25 smithi132 ceph-mon[29178]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:25.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:25 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1216828149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de1bb55b-fced-40bd-b2c3-81a227b2e891"}]: dispatch 2024-04-23T18:29:25.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:25 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1216828149' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de1bb55b-fced-40bd-b2c3-81a227b2e891"}]': finished 2024-04-23T18:29:25.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:25 smithi132 ceph-mon[29178]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T18:29:25.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:25 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:25.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:25 smithi148 ceph-mon[32311]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:25.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:25 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1216828149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de1bb55b-fced-40bd-b2c3-81a227b2e891"}]: dispatch 2024-04-23T18:29:25.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:25 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1216828149' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de1bb55b-fced-40bd-b2c3-81a227b2e891"}]': finished 2024-04-23T18:29:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:25 smithi148 ceph-mon[32311]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T18:29:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:25 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:25 smithi157 ceph-mon[32580]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:25 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1216828149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de1bb55b-fced-40bd-b2c3-81a227b2e891"}]: dispatch 2024-04-23T18:29:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:25 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1216828149' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de1bb55b-fced-40bd-b2c3-81a227b2e891"}]': finished 2024-04-23T18:29:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:25 smithi157 ceph-mon[32580]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T18:29:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:25 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:26.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:26 smithi132 ceph-mon[29178]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T18:29:26.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:26 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3373664505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:29:26.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:26 smithi148 ceph-mon[32311]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T18:29:26.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:26 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3373664505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:29:26.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:26 smithi157 ceph-mon[32580]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T18:29:26.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:26 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3373664505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:29:27.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:27 smithi132 ceph-mon[29178]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:27.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:27 smithi148 ceph-mon[32311]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:27.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:27 smithi157 ceph-mon[32580]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:29.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:29 smithi148 ceph-mon[32311]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:29.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:29 smithi157 ceph-mon[32580]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:29.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:29 smithi132 ceph-mon[29178]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:31.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:31 smithi132 ceph-mon[29178]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:31.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:31 smithi148 ceph-mon[32311]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:31.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:31 smithi157 ceph-mon[32580]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:32.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T18:29:32.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:32.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T18:29:32.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:32.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T18:29:32.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:33 smithi132 ceph-mon[29178]: Deploying daemon osd.3 on smithi132 2024-04-23T18:29:33.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:33 smithi132 ceph-mon[29178]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:33.441 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:33 smithi157 ceph-mon[32580]: Deploying daemon osd.3 on smithi132 2024-04-23T18:29:33.441 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:33 smithi157 ceph-mon[32580]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:33.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:33 smithi148 ceph-mon[32311]: Deploying daemon osd.3 on smithi132 2024-04-23T18:29:33.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:33 smithi148 ceph-mon[32311]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:35.388 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:35 smithi132 ceph-mon[29178]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:35.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:35 smithi148 ceph-mon[32311]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:35.912 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:35 smithi157 ceph-mon[32580]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:37.228 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:37 smithi132 ceph-mon[29178]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:37.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:37 smithi148 ceph-mon[32311]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:37.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:37 smithi157 ceph-mon[32580]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:39.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:39.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:39.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:39.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:38 smithi132 ceph-mon[29178]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:39.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:39.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:39.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:39.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:38 smithi148 ceph-mon[32311]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:39.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:39.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:39.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:39.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:38 smithi157 ceph-mon[32580]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:41.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.231+0000 7f188a7fc700 1 -- 172.21.15.132:0/2508009000 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f189406af20 con 0x7f18800a3bb0 2024-04-23T18:29:41.225 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 3 on host 'smithi132' 2024-04-23T18:29:41.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.234+0000 7f189b55f700 1 -- 172.21.15.132:0/2508009000 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f18800a3bb0 msgr2=0x7f18800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.234+0000 7f189b55f700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f18800a3bb0 0x7f18800a6070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1884005950 tx=0x7f188400b410 comp rx=0 tx=0).stop 2024-04-23T18:29:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.234+0000 7f189b55f700 1 -- 172.21.15.132:0/2508009000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18941019a0 msgr2=0x7f1894197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.234+0000 7f189b55f700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18941019a0 0x7f1894197ac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1890007b80 tx=0x7f1890007e90 comp rx=0 tx=0).stop 2024-04-23T18:29:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.234+0000 7f189b55f700 1 -- 172.21.15.132:0/2508009000 shutdown_connections 2024-04-23T18:29:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.234+0000 7f189b55f700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1894103540 0x7f189419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.234+0000 7f189b55f700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f18800a3bb0 0x7f18800a6070 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.234+0000 7f189b55f700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18941019a0 0x7f1894197ac0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.234+0000 7f189b55f700 1 --2- 172.21.15.132:0/2508009000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894102ba0 0x7f1894198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.234+0000 7f189b55f700 1 -- 172.21.15.132:0/2508009000 >> 172.21.15.132:0/2508009000 conn(0x7f18940fcf40 msgr2=0x7f1894105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.234+0000 7f189b55f700 1 -- 172.21.15.132:0/2508009000 shutdown_connections 2024-04-23T18:29:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:29:41.234+0000 7f189b55f700 1 -- 172.21.15.132:0/2508009000 wait complete. 2024-04-23T18:29:41.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:41 smithi148 ceph-mon[32311]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:41.528 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:41 smithi132 ceph-mon[29178]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:41 smithi157 ceph-mon[32580]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:41.812 DEBUG:teuthology.orchestra.run.smithi132:osd.3> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.3.service 2024-04-23T18:29:41.815 INFO:tasks.cephadm:Deploying osd.4 on smithi148 with /dev/vg_nvme/lv_3... 2024-04-23T18:29:41.815 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T18:29:42.063 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:29:42.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:42 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:42.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:42 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:42.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:42 smithi148 ceph-mon[32311]: from='osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T18:29:42.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:42 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:42.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:42 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:42.531 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:42 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:42.531 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:42 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:42.531 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:42 smithi132 ceph-mon[29178]: from='osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T18:29:42.531 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:42 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:42.531 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:42 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:42 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:42 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:42 smithi157 ceph-mon[32580]: from='osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T18:29:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:42 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:42 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:43.481 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:43 smithi148 ceph-mon[32311]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:43.481 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:43 smithi148 ceph-mon[32311]: from='osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T18:29:43.481 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:43 smithi148 ceph-mon[32311]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T18:29:43.481 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:43 smithi148 ceph-mon[32311]: from='osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-04-23T18:29:43.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:43 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:43.520 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:43 smithi132 ceph-mon[29178]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:43.520 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:43 smithi132 ceph-mon[29178]: from='osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T18:29:43.521 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:43 smithi132 ceph-mon[29178]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T18:29:43.521 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:43 smithi132 ceph-mon[29178]: from='osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-04-23T18:29:43.521 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:43 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:43.521 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:29:43 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3[50631]: 2024-04-23T18:29:43.230+0000 7f9348f9d700 -1 osd.3 0 waiting for initial osdmap 2024-04-23T18:29:43.521 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:29:43 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3[50631]: 2024-04-23T18:29:43.237+0000 7f9344d90700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:29:43.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:43 smithi157 ceph-mon[32580]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:43.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:43 smithi157 ceph-mon[32580]: from='osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T18:29:43.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:43 smithi157 ceph-mon[32580]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T18:29:43.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:43 smithi157 ceph-mon[32580]: from='osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-04-23T18:29:43.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:43 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:44.066 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T18:29:44.066 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T18:29:44.066 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-04-23T18:29:44.066 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-04-23T18:29:44.067 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112904 s, 929 MB/s 2024-04-23T18:29:44.067 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-04-23T18:29:44.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:44 smithi148 ceph-mon[32311]: from='osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-04-23T18:29:44.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:44 smithi148 ceph-mon[32311]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T18:29:44.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:44 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:44.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:44 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:44.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:44 smithi148 ceph-mon[32311]: osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155] boot 2024-04-23T18:29:44.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:44 smithi148 ceph-mon[32311]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T18:29:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:44 smithi157 ceph-mon[32580]: from='osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-04-23T18:29:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:44 smithi157 ceph-mon[32580]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T18:29:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:44 smithi157 ceph-mon[32580]: osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155] boot 2024-04-23T18:29:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:44 smithi157 ceph-mon[32580]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T18:29:44.604 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:44 smithi132 ceph-mon[29178]: from='osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-04-23T18:29:44.604 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:44 smithi132 ceph-mon[29178]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T18:29:44.605 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:44.605 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:44.605 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:44 smithi132 ceph-mon[29178]: osd.3 [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155] boot 2024-04-23T18:29:44.605 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:44 smithi132 ceph-mon[29178]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T18:29:44.764 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_3 2024-04-23T18:29:45.004 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:29:45.413 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:45 smithi148 ceph-mon[32311]: purged_snaps scrub starts 2024-04-23T18:29:45.413 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:45 smithi148 ceph-mon[32311]: purged_snaps scrub ok 2024-04-23T18:29:45.414 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:45 smithi148 ceph-mon[32311]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:45.414 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:45 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:45.414 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:45 smithi148 ceph-mon[32311]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T18:29:45.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:45 smithi157 ceph-mon[32580]: purged_snaps scrub starts 2024-04-23T18:29:45.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:45 smithi157 ceph-mon[32580]: purged_snaps scrub ok 2024-04-23T18:29:45.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:45 smithi157 ceph-mon[32580]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:45.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:45 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:45.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:45 smithi157 ceph-mon[32580]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T18:29:45.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:45 smithi132 ceph-mon[29178]: purged_snaps scrub starts 2024-04-23T18:29:45.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:45 smithi132 ceph-mon[29178]: purged_snaps scrub ok 2024-04-23T18:29:45.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:45 smithi132 ceph-mon[29178]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:29:45.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:45 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:29:45.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:45 smithi132 ceph-mon[29178]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T18:29:46.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: Detected new or changed devices on smithi132 2024-04-23T18:29:46.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:29:46.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:29:46.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi132 to 8544M 2024-04-23T18:29:46.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:46.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:46.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:46.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:46.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:46.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:29:46.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:46 smithi148 ceph-mon[32311]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T18:29:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: Detected new or changed devices on smithi132 2024-04-23T18:29:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:29:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:29:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi132 to 8544M 2024-04-23T18:29:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:46.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:46.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:46.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:46.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:29:46.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:46 smithi132 ceph-mon[29178]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T18:29:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.671+0000 7f9e5cb70700 1 -- 172.21.15.148:0/2768629924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e58103540 msgr2=0x7f9e58109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.671+0000 7f9e5cb70700 1 --2- 172.21.15.148:0/2768629924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e58103540 0x7f9e58109df0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000b0d0 tx=0x7f9e5000b3e0 comp rx=0 tx=0).stop 2024-04-23T18:29:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.671+0000 7f9e5cb70700 1 -- 172.21.15.148:0/2768629924 shutdown_connections 2024-04-23T18:29:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.671+0000 7f9e5cb70700 1 --2- 172.21.15.148:0/2768629924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e58103540 0x7f9e58109df0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.671+0000 7f9e5cb70700 1 --2- 172.21.15.148:0/2768629924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e581019a0 0x7f9e58101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.671+0000 7f9e5cb70700 1 --2- 172.21.15.148:0/2768629924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e58102ba0 0x7f9e58103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.671+0000 7f9e5cb70700 1 -- 172.21.15.148:0/2768629924 >> 172.21.15.148:0/2768629924 conn(0x7f9e580fcf40 msgr2=0x7f9e580ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.671+0000 7f9e5cb70700 1 -- 172.21.15.148:0/2768629924 shutdown_connections 2024-04-23T18:29:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.671+0000 7f9e5cb70700 1 -- 172.21.15.148:0/2768629924 wait complete. 2024-04-23T18:29:46.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.672+0000 7f9e5cb70700 1 Processor -- start 2024-04-23T18:29:46.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.672+0000 7f9e5cb70700 1 -- start start 2024-04-23T18:29:46.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.672+0000 7f9e5cb70700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e581019a0 0x7f9e58198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:46.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.672+0000 7f9e5cb70700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e58102ba0 0x7f9e58199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:46.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.672+0000 7f9e5cb70700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e58103540 0x7f9e581a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.672+0000 7f9e5cb70700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5810d4b0 con 0x7f9e581019a0 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.672+0000 7f9e5cb70700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5810d330 con 0x7f9e58102ba0 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.672+0000 7f9e5cb70700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5810d630 con 0x7f9e58103540 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.672+0000 7f9e55d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e58102ba0 0x7f9e58199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.672+0000 7f9e55d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e58102ba0 0x7f9e58199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47960/0 (socket says 172.21.15.148:47960) 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.672+0000 7f9e55d9b700 1 -- 172.21.15.148:0/1065182806 learned_addr learned my addr 172.21.15.148:0/1065182806 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.672+0000 7f9e5659c700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e581019a0 0x7f9e58198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e56d9d700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e58103540 0x7f9e581a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e55d9b700 1 -- 172.21.15.148:0/1065182806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e58103540 msgr2=0x7f9e581a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e55d9b700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e58103540 0x7f9e581a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e55d9b700 1 -- 172.21.15.148:0/1065182806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e581019a0 msgr2=0x7f9e58198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e55d9b700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e581019a0 0x7f9e58198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e55d9b700 1 -- 172.21.15.148:0/1065182806 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e581a0820 con 0x7f9e58102ba0 2024-04-23T18:29:46.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e5659c700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e581019a0 0x7f9e58198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e56d9d700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e58103540 0x7f9e581a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e55d9b700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e58102ba0 0x7f9e58199100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800c920 tx=0x7f9e4800cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e477fe700 1 -- 172.21.15.148:0/1065182806 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48007b00 con 0x7f9e58102ba0 2024-04-23T18:29:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e5cb70700 1 -- 172.21.15.148:0/1065182806 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e581a0b10 con 0x7f9e58102ba0 2024-04-23T18:29:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e477fe700 1 -- 172.21.15.148:0/1065182806 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e48004ce0 con 0x7f9e58102ba0 2024-04-23T18:29:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e477fe700 1 -- 172.21.15.148:0/1065182806 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e480055f0 con 0x7f9e58102ba0 2024-04-23T18:29:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.673+0000 7f9e5cb70700 1 -- 172.21.15.148:0/1065182806 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e581a1050 con 0x7f9e58102ba0 2024-04-23T18:29:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.674+0000 7f9e5cb70700 1 -- 172.21.15.148:0/1065182806 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e58069730 con 0x7f9e58102ba0 2024-04-23T18:29:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.675+0000 7f9e477fe700 1 -- 172.21.15.148:0/1065182806 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9e48004750 con 0x7f9e58102ba0 2024-04-23T18:29:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.675+0000 7f9e477fe700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9e400a3c40 0x7f9e400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.676+0000 7f9e477fe700 1 -- 172.21.15.148:0/1065182806 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f9e480cf190 con 0x7f9e58102ba0 2024-04-23T18:29:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.676+0000 7f9e5659c700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9e400a3c40 0x7f9e400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.676+0000 7f9e5659c700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9e400a3c40 0x7f9e400a6100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e58102a00 tx=0x7f9e4c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:46.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.680+0000 7f9e477fe700 1 -- 172.21.15.148:0/1065182806 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e48014030 con 0x7f9e58102ba0 2024-04-23T18:29:46.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: Detected new or changed devices on smithi132 2024-04-23T18:29:46.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:29:46.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:29:46.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi132 to 8544M 2024-04-23T18:29:46.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:46.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:46.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:46.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:46.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:46.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:29:46.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:29:46.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:46 smithi157 ceph-mon[32580]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T18:29:46.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:29:46.885+0000 7f9e5cb70700 1 -- 172.21.15.148:0/1065182806 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9e580ff0e0 con 0x7f9e400a3c40 2024-04-23T18:29:47.731 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:47 smithi148 ceph-mon[32311]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:29:47.731 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:47 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:29:47.731 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:47 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:29:47.731 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:47 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:47.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:47 smithi157 ceph-mon[32580]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:29:47.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:47 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:29:47.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:47 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:29:47.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:47 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:47.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:47 smithi132 ceph-mon[29178]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:29:47.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:47 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:29:47.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:47 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:29:47.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:47 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:48.759 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:48 smithi148 ceph-mon[32311]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T18:29:48.759 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:48 smithi148 ceph-mon[32311]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:29:48.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:48 smithi157 ceph-mon[32580]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T18:29:48.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:48 smithi157 ceph-mon[32580]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:29:48.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:48 smithi132 ceph-mon[29178]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T18:29:48.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:48 smithi132 ceph-mon[29178]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:29:49.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:49 smithi148 ceph-mon[32311]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T18:29:49.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:49 smithi157 ceph-mon[32580]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T18:29:49.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:49 smithi132 ceph-mon[29178]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T18:29:50.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:50 smithi148 ceph-mon[32311]: from='client.? 172.21.15.148:0/2879532521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e84996c-3580-48d6-b38f-ad1cd3217e70"}]: dispatch 2024-04-23T18:29:50.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:50 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e84996c-3580-48d6-b38f-ad1cd3217e70"}]: dispatch 2024-04-23T18:29:50.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:50 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e84996c-3580-48d6-b38f-ad1cd3217e70"}]': finished 2024-04-23T18:29:50.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:50 smithi148 ceph-mon[32311]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T18:29:50.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:50 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:29:50.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:50 smithi148 ceph-mon[32311]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:29:50.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:50 smithi148 ceph-mon[32311]: from='client.? 172.21.15.148:0/3612313004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:29:50.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:50 smithi157 ceph-mon[32580]: from='client.? 172.21.15.148:0/2879532521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e84996c-3580-48d6-b38f-ad1cd3217e70"}]: dispatch 2024-04-23T18:29:50.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:50 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e84996c-3580-48d6-b38f-ad1cd3217e70"}]: dispatch 2024-04-23T18:29:50.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:50 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e84996c-3580-48d6-b38f-ad1cd3217e70"}]': finished 2024-04-23T18:29:50.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:50 smithi157 ceph-mon[32580]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T18:29:50.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:50 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:29:50.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:50 smithi157 ceph-mon[32580]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:29:50.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:50 smithi157 ceph-mon[32580]: from='client.? 172.21.15.148:0/3612313004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:29:50.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:50 smithi132 ceph-mon[29178]: from='client.? 172.21.15.148:0/2879532521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e84996c-3580-48d6-b38f-ad1cd3217e70"}]: dispatch 2024-04-23T18:29:50.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:50 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e84996c-3580-48d6-b38f-ad1cd3217e70"}]: dispatch 2024-04-23T18:29:50.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:50 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e84996c-3580-48d6-b38f-ad1cd3217e70"}]': finished 2024-04-23T18:29:50.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:50 smithi132 ceph-mon[29178]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T18:29:50.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:50 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:29:50.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:50 smithi132 ceph-mon[29178]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:29:50.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:50 smithi132 ceph-mon[29178]: from='client.? 172.21.15.148:0/3612313004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:29:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:53 smithi148 ceph-mon[32311]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-23T18:29:53.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:53 smithi157 ceph-mon[32580]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-23T18:29:53.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:53 smithi132 ceph-mon[29178]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-23T18:29:55.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:55 smithi148 ceph-mon[32311]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-23T18:29:55.415 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:55 smithi132 ceph-mon[29178]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-23T18:29:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:55 smithi157 ceph-mon[32580]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-23T18:29:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:57 smithi148 ceph-mon[32311]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:29:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T18:29:57.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:57 smithi157 ceph-mon[32580]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:29:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T18:29:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:57.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:57 smithi132 ceph-mon[29178]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:29:57.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T18:29:57.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:58.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:58 smithi148 ceph-mon[32311]: Deploying daemon osd.4 on smithi148 2024-04-23T18:29:58.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:58 smithi157 ceph-mon[32580]: Deploying daemon osd.4 on smithi148 2024-04-23T18:29:58.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:58 smithi132 ceph-mon[29178]: Deploying daemon osd.4 on smithi148 2024-04-23T18:29:59.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:29:59 smithi148 ceph-mon[32311]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-23T18:29:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:29:59 smithi157 ceph-mon[32580]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-23T18:29:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:29:59 smithi132 ceph-mon[29178]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-23T18:30:00.398 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:00 smithi148 ceph-mon[32311]: overall HEALTH_OK 2024-04-23T18:30:00.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:00 smithi157 ceph-mon[32580]: overall HEALTH_OK 2024-04-23T18:30:00.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:00 smithi132 ceph-mon[29178]: overall HEALTH_OK 2024-04-23T18:30:01.402 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:01 smithi148 ceph-mon[32311]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-23T18:30:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:01 smithi157 ceph-mon[32580]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-23T18:30:01.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:01 smithi132 ceph-mon[29178]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-23T18:30:03.262 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:03 smithi148 ceph-mon[32311]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:30:03.262 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:03 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:03.262 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:03 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:03.262 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:03 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:03.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:03 smithi157 ceph-mon[32580]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:30:03.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:03 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:03.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:03 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:03.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:03 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:03.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:03 smithi132 ceph-mon[29178]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:30:03.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:03 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:03.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:03 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:03.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:03 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:05.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:05 smithi148 ceph-mon[32311]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:30:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:05 smithi157 ceph-mon[32580]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:30:05.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:05 smithi132 ceph-mon[29178]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:30:06.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.202+0000 7f9e477fe700 1 -- 172.21.15.148:0/1065182806 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9e580ff0e0 con 0x7f9e400a3c40 2024-04-23T18:30:06.195 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 4 on host 'smithi148' 2024-04-23T18:30:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.204+0000 7f9e5cb70700 1 -- 172.21.15.148:0/1065182806 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9e400a3c40 msgr2=0x7f9e400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.204+0000 7f9e5cb70700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9e400a3c40 0x7f9e400a6100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e58102a00 tx=0x7f9e4c008040 comp rx=0 tx=0).stop 2024-04-23T18:30:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.204+0000 7f9e5cb70700 1 -- 172.21.15.148:0/1065182806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e58102ba0 msgr2=0x7f9e58199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.204+0000 7f9e5cb70700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e58102ba0 0x7f9e58199100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800c920 tx=0x7f9e4800cce0 comp rx=0 tx=0).stop 2024-04-23T18:30:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.205+0000 7f9e5cb70700 1 -- 172.21.15.148:0/1065182806 shutdown_connections 2024-04-23T18:30:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.205+0000 7f9e5cb70700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e58103540 0x7f9e581a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.205+0000 7f9e5cb70700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9e400a3c40 0x7f9e400a6100 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.205+0000 7f9e5cb70700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e581019a0 0x7f9e58198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.205+0000 7f9e5cb70700 1 --2- 172.21.15.148:0/1065182806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e58102ba0 0x7f9e58199100 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.205+0000 7f9e5cb70700 1 -- 172.21.15.148:0/1065182806 >> 172.21.15.148:0/1065182806 conn(0x7f9e580fcf40 msgr2=0x7f9e580fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.205+0000 7f9e5cb70700 1 -- 172.21.15.148:0/1065182806 shutdown_connections 2024-04-23T18:30:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:30:06.205+0000 7f9e5cb70700 1 -- 172.21.15.148:0/1065182806 wait complete. 2024-04-23T18:30:06.816 DEBUG:teuthology.orchestra.run.smithi148:osd.4> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.4.service 2024-04-23T18:30:06.819 INFO:tasks.cephadm:Deploying osd.5 on smithi157 with /dev/vg_nvme/lv_3... 2024-04-23T18:30:06.819 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T18:30:07.067 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:30:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:07 smithi148 ceph-mon[32311]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:30:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:07 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:07 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:07 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:07 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:07 smithi148 ceph-mon[32311]: from='osd.4 [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:30:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:07 smithi148 ceph-mon[32311]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:30:07.431 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:07 smithi157 ceph-mon[32580]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:30:07.431 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:07 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:07.431 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:07 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:07.431 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:07 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:07.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:07 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:07.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:07 smithi157 ceph-mon[32580]: from='osd.4 [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:30:07.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:07 smithi157 ceph-mon[32580]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:30:07.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:07 smithi132 ceph-mon[29178]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:30:07.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:07 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:07.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:07 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:07 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:07 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:07 smithi132 ceph-mon[29178]: from='osd.4 [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:30:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:07 smithi132 ceph-mon[29178]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:30:08.350 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:08 smithi148 ceph-mon[32311]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T18:30:08.350 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:08 smithi148 ceph-mon[32311]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T18:30:08.350 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:08 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:08.350 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:08 smithi148 ceph-mon[32311]: from='osd.4 [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:30:08.351 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:08 smithi148 ceph-mon[32311]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:30:08.351 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:30:08 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-23T18:30:08.234+0000 7ffbac55e700 -1 osd.4 0 waiting for initial osdmap 2024-04-23T18:30:08.351 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:30:08 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-23T18:30:08.244+0000 7ffba8351700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:30:08.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:08 smithi157 ceph-mon[32580]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T18:30:08.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:08 smithi157 ceph-mon[32580]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T18:30:08.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:08 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:08.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:08 smithi157 ceph-mon[32580]: from='osd.4 [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:30:08.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:08 smithi157 ceph-mon[32580]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:30:08.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:08 smithi132 ceph-mon[29178]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T18:30:08.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:08 smithi132 ceph-mon[29178]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T18:30:08.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:08 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:08.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:08 smithi132 ceph-mon[29178]: from='osd.4 [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:30:08.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:08 smithi132 ceph-mon[29178]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:30:08.979 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T18:30:08.979 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T18:30:08.979 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-04-23T18:30:08.979 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-04-23T18:30:08.979 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00818189 s, 1.3 GB/s 2024-04-23T18:30:08.980 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-04-23T18:30:09.504 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:09 smithi148 ceph-mon[32311]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:30:09.504 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:09 smithi148 ceph-mon[32311]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-23T18:30:09.504 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:09 smithi148 ceph-mon[32311]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T18:30:09.504 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:09.504 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:09.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:09 smithi157 ceph-mon[32580]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:30:09.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:09 smithi157 ceph-mon[32580]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-23T18:30:09.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:09 smithi157 ceph-mon[32580]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T18:30:09.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:09.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:09.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:09 smithi132 ceph-mon[29178]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:30:09.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:09 smithi132 ceph-mon[29178]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-23T18:30:09.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:09 smithi132 ceph-mon[29178]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T18:30:09.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:09.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:09.614 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_3 2024-04-23T18:30:09.863 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:30:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: purged_snaps scrub starts 2024-04-23T18:30:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: purged_snaps scrub ok 2024-04-23T18:30:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: osd.4 [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121] boot 2024-04-23T18:30:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T18:30:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: Detected new or changed devices on smithi148 2024-04-23T18:30:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:10.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:10.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi148 to 8544M 2024-04-23T18:30:10.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:10.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:10.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:30:10.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:10.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:10.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:10.516 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: purged_snaps scrub starts 2024-04-23T18:30:10.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: purged_snaps scrub ok 2024-04-23T18:30:10.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:10.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: osd.4 [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121] boot 2024-04-23T18:30:10.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T18:30:10.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: Detected new or changed devices on smithi148 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi148 to 8544M 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:10.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:10.542 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: purged_snaps scrub starts 2024-04-23T18:30:10.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: purged_snaps scrub ok 2024-04-23T18:30:10.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:10.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: osd.4 [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121] boot 2024-04-23T18:30:10.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T18:30:10.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:30:10.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: Detected new or changed devices on smithi148 2024-04-23T18:30:10.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi148 to 8544M 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:10.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:11.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.429+0000 7fc24e091700 1 -- 172.21.15.157:0/2146368947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc248103530 msgr2=0x7fc248109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:11.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.429+0000 7fc24e091700 1 --2- 172.21.15.157:0/2146368947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc248103530 0x7fc248109de0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc24000b0a0 tx=0x7fc24000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:30:11.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.429+0000 7fc24e091700 1 -- 172.21.15.157:0/2146368947 shutdown_connections 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.429+0000 7fc24e091700 1 --2- 172.21.15.157:0/2146368947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc248103530 0x7fc248109de0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.429+0000 7fc24e091700 1 --2- 172.21.15.157:0/2146368947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc248101990 0x7fc248101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.429+0000 7fc24e091700 1 --2- 172.21.15.157:0/2146368947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc248102b90 0x7fc248102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.429+0000 7fc24e091700 1 -- 172.21.15.157:0/2146368947 >> 172.21.15.157:0/2146368947 conn(0x7fc2480fcf30 msgr2=0x7fc2480ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.429+0000 7fc24e091700 1 -- 172.21.15.157:0/2146368947 shutdown_connections 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.429+0000 7fc24e091700 1 -- 172.21.15.157:0/2146368947 wait complete. 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc24e091700 1 Processor -- start 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc24e091700 1 -- start start 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc24e091700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc248101990 0x7fc248197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc24e091700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc248102b90 0x7fc248197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc24e091700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc248103530 0x7fc24819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc24e091700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc24810d430 con 0x7fc248102b90 2024-04-23T18:30:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc24e091700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc24810d2b0 con 0x7fc248101990 2024-04-23T18:30:11.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc24e091700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc24810d5b0 con 0x7fc248103530 2024-04-23T18:30:11.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc247fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc248103530 0x7fc24819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:11.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc247fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc248103530 0x7fc24819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44500/0 (socket says 172.21.15.157:44500) 2024-04-23T18:30:11.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc2477fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc248101990 0x7fc248197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:11.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc247fff700 1 -- 172.21.15.157:0/2654587337 learned_addr learned my addr 172.21.15.157:0/2654587337 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-23T18:30:11.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.430+0000 7fc23effd700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc248102b90 0x7fc248197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:11.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.431+0000 7fc247fff700 1 -- 172.21.15.157:0/2654587337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc248101990 msgr2=0x7fc248197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:11.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.431+0000 7fc247fff700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc248101990 0x7fc248197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:11.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.431+0000 7fc247fff700 1 -- 172.21.15.157:0/2654587337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc248102b90 msgr2=0x7fc248197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:11.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.431+0000 7fc247fff700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc248102b90 0x7fc248197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:11.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.431+0000 7fc247fff700 1 -- 172.21.15.157:0/2654587337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24819f6b0 con 0x7fc248103530 2024-04-23T18:30:11.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.431+0000 7fc247fff700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc248103530 0x7fc24819f010 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc240004e40 tx=0x7fc240004e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:11.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.431+0000 7fc2457fa700 1 -- 172.21.15.157:0/2654587337 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240021070 con 0x7fc248103530 2024-04-23T18:30:11.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.431+0000 7fc24e091700 1 -- 172.21.15.157:0/2654587337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24819f940 con 0x7fc248103530 2024-04-23T18:30:11.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.431+0000 7fc2457fa700 1 -- 172.21.15.157:0/2654587337 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc240007c00 con 0x7fc248103530 2024-04-23T18:30:11.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.431+0000 7fc2457fa700 1 -- 172.21.15.157:0/2654587337 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240004730 con 0x7fc248103530 2024-04-23T18:30:11.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.431+0000 7fc24e091700 1 -- 172.21.15.157:0/2654587337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc24819fdf0 con 0x7fc248103530 2024-04-23T18:30:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.432+0000 7fc24e091700 1 -- 172.21.15.157:0/2654587337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc248069730 con 0x7fc248103530 2024-04-23T18:30:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.433+0000 7fc2457fa700 1 -- 172.21.15.157:0/2654587337 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc240003a00 con 0x7fc248103530 2024-04-23T18:30:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.433+0000 7fc2457fa700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc2340a3c40 0x7fc2340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.434+0000 7fc2457fa700 1 -- 172.21.15.157:0/2654587337 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fc24000f020 con 0x7fc248103530 2024-04-23T18:30:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.434+0000 7fc2477fe700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc2340a3c40 0x7fc2340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.434+0000 7fc2477fe700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc2340a3c40 0x7fc2340a6100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc2481029f0 tx=0x7fc23800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:11.430 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.438+0000 7fc2457fa700 1 -- 172.21.15.157:0/2654587337 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc24002a020 con 0x7fc248103530 2024-04-23T18:30:11.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:11 smithi148 ceph-mon[32311]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T18:30:11.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:11 smithi157 ceph-mon[32580]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T18:30:11.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:11 smithi132 ceph-mon[29178]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T18:30:11.638 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:11.645+0000 7fc24e091700 1 -- 172.21.15.157:0/2654587337 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc248062f40 con 0x7fc2340a3c40 2024-04-23T18:30:12.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:12 smithi148 ceph-mon[32311]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:30:12.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:30:12.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:30:12.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:12.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:12 smithi148 ceph-mon[32311]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:12.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:12 smithi157 ceph-mon[32580]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:30:12.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:30:12.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:30:12.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:12.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:12 smithi157 ceph-mon[32580]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:12.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:12 smithi132 ceph-mon[29178]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:30:12.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:30:12.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:30:12.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:12.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:12 smithi132 ceph-mon[29178]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:15.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:15 smithi148 ceph-mon[32311]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:15 smithi148 ceph-mon[32311]: from='client.? 172.21.15.157:0/2492124944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f06f9283-8099-4922-8e1c-f28c4ea57a64"}]: dispatch 2024-04-23T18:30:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:15 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f06f9283-8099-4922-8e1c-f28c4ea57a64"}]: dispatch 2024-04-23T18:30:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:15 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f06f9283-8099-4922-8e1c-f28c4ea57a64"}]': finished 2024-04-23T18:30:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:15 smithi148 ceph-mon[32311]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T18:30:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:15 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:30:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:15 smithi157 ceph-mon[32580]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:15 smithi157 ceph-mon[32580]: from='client.? 172.21.15.157:0/2492124944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f06f9283-8099-4922-8e1c-f28c4ea57a64"}]: dispatch 2024-04-23T18:30:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:15 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f06f9283-8099-4922-8e1c-f28c4ea57a64"}]: dispatch 2024-04-23T18:30:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:15 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f06f9283-8099-4922-8e1c-f28c4ea57a64"}]': finished 2024-04-23T18:30:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:15 smithi157 ceph-mon[32580]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T18:30:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:15 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:30:15.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:15 smithi132 ceph-mon[29178]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:15 smithi132 ceph-mon[29178]: from='client.? 172.21.15.157:0/2492124944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f06f9283-8099-4922-8e1c-f28c4ea57a64"}]: dispatch 2024-04-23T18:30:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:15 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f06f9283-8099-4922-8e1c-f28c4ea57a64"}]: dispatch 2024-04-23T18:30:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:15 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f06f9283-8099-4922-8e1c-f28c4ea57a64"}]': finished 2024-04-23T18:30:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:15 smithi132 ceph-mon[29178]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T18:30:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:15 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:30:16.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:16 smithi148 ceph-mon[32311]: from='client.? 172.21.15.157:0/1284348456' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:30:16.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:16 smithi157 ceph-mon[32580]: from='client.? 172.21.15.157:0/1284348456' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:30:16.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:16 smithi132 ceph-mon[29178]: from='client.? 172.21.15.157:0/1284348456' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:30:17.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:17 smithi148 ceph-mon[32311]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:17.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:17 smithi157 ceph-mon[32580]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:17.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:17 smithi132 ceph-mon[29178]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:19.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:19 smithi157 ceph-mon[32580]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:19.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:19 smithi148 ceph-mon[32311]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:19.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:19 smithi132 ceph-mon[29178]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:21.450 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:21 smithi157 ceph-mon[32580]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:21.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:21 smithi148 ceph-mon[32311]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:21.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:21 smithi132 ceph-mon[29178]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:22.418 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:22 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T18:30:22.419 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:22 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:22.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:22 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T18:30:22.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:22 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:22.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:22 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T18:30:22.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:22 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:23.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:23 smithi157 ceph-mon[32580]: Deploying daemon osd.5 on smithi157 2024-04-23T18:30:23.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:23 smithi157 ceph-mon[32580]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:23.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:23 smithi148 ceph-mon[32311]: Deploying daemon osd.5 on smithi157 2024-04-23T18:30:23.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:23 smithi148 ceph-mon[32311]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:23.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:23 smithi132 ceph-mon[29178]: Deploying daemon osd.5 on smithi157 2024-04-23T18:30:23.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:23 smithi132 ceph-mon[29178]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:25.414 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:25 smithi132 ceph-mon[29178]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:25.454 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:25 smithi157 ceph-mon[32580]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:25.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:25 smithi148 ceph-mon[32311]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:27.247 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:27 smithi157 ceph-mon[32580]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:27.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:27 smithi148 ceph-mon[32311]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:27.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:27 smithi132 ceph-mon[29178]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:28.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:28.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:28.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:28.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:28 smithi157 ceph-mon[32580]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:28.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:28.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:28 smithi132 ceph-mon[29178]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:29.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:29.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:29.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:29.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:28 smithi148 ceph-mon[32311]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:31.394 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:31 smithi157 ceph-mon[32580]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:31.394 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:31.394 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:31.394 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:31.395 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:31.395 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:31 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:31.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.461+0000 7fc2457fa700 1 -- 172.21.15.157:0/2654587337 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc248062f40 con 0x7fc2340a3c40 2024-04-23T18:30:31.455 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 5 on host 'smithi157' 2024-04-23T18:30:31.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.464+0000 7fc24e091700 1 -- 172.21.15.157:0/2654587337 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc2340a3c40 msgr2=0x7fc2340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:31.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.464+0000 7fc24e091700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc2340a3c40 0x7fc2340a6100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc2481029f0 tx=0x7fc23800b410 comp rx=0 tx=0).stop 2024-04-23T18:30:31.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.464+0000 7fc24e091700 1 -- 172.21.15.157:0/2654587337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc248103530 msgr2=0x7fc24819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:31.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.464+0000 7fc24e091700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc248103530 0x7fc24819f010 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc240004e40 tx=0x7fc240004e70 comp rx=0 tx=0).stop 2024-04-23T18:30:31.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.464+0000 7fc24e091700 1 -- 172.21.15.157:0/2654587337 shutdown_connections 2024-04-23T18:30:31.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.464+0000 7fc24e091700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc248103530 0x7fc24819f010 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:31.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.464+0000 7fc24e091700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc248101990 0x7fc248197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:31.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.464+0000 7fc24e091700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc2340a3c40 0x7fc2340a6100 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:31.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.464+0000 7fc24e091700 1 --2- 172.21.15.157:0/2654587337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc248102b90 0x7fc248197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:31.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.464+0000 7fc24e091700 1 -- 172.21.15.157:0/2654587337 >> 172.21.15.157:0/2654587337 conn(0x7fc2480fcf30 msgr2=0x7fc248105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:31.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.464+0000 7fc24e091700 1 -- 172.21.15.157:0/2654587337 shutdown_connections 2024-04-23T18:30:31.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:30:31.464+0000 7fc24e091700 1 -- 172.21.15.157:0/2654587337 wait complete. 2024-04-23T18:30:31.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:31 smithi148 ceph-mon[32311]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:31.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:31.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:31.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:31.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:31.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:31 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:31.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:31 smithi132 ceph-mon[29178]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:31.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:31.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:31.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:31.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:31.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:31 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:32.108 DEBUG:teuthology.orchestra.run.smithi157:osd.5> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.5.service 2024-04-23T18:30:32.110 INFO:tasks.cephadm:Deploying osd.6 on smithi132 with /dev/vg_nvme/lv_2... 2024-04-23T18:30:32.110 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T18:30:32.359 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:30:32.401 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:32 smithi157 ceph-mon[32580]: from='osd.5 [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:30:32.401 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:32 smithi157 ceph-mon[32580]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:30:32.401 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:32.401 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:32.402 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:32.481 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:32 smithi132 ceph-mon[29178]: from='osd.5 [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:30:32.482 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:32 smithi132 ceph-mon[29178]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:30:32.482 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:32.482 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:32.482 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:32.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:32 smithi148 ceph-mon[32311]: from='osd.5 [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:30:32.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:32 smithi148 ceph-mon[32311]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:30:32.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:32.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:32.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:33.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:33 smithi157 ceph-mon[32580]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:33.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:33 smithi157 ceph-mon[32580]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T18:30:33.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:33 smithi157 ceph-mon[32580]: from='osd.5 [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:30:33.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:33 smithi157 ceph-mon[32580]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T18:30:33.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:33 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:30:33.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:33 smithi157 ceph-mon[32580]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:30:33.291 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:30:33 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:30:33.155+0000 7f7584dd6700 -1 osd.5 0 waiting for initial osdmap 2024-04-23T18:30:33.291 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:30:33 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:30:33.161+0000 7f7580bc9700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:30:33.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:33 smithi148 ceph-mon[32311]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:33.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:33 smithi148 ceph-mon[32311]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T18:30:33.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:33 smithi148 ceph-mon[32311]: from='osd.5 [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:30:33.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:33 smithi148 ceph-mon[32311]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T18:30:33.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:33 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:30:33.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:33 smithi148 ceph-mon[32311]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:30:33.567 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:33 smithi132 ceph-mon[29178]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:33.568 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:33 smithi132 ceph-mon[29178]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T18:30:33.568 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:33 smithi132 ceph-mon[29178]: from='osd.5 [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:30:33.568 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:33 smithi132 ceph-mon[29178]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T18:30:33.568 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:33 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:30:33.568 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:33 smithi132 ceph-mon[29178]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:30:34.444 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:34 smithi157 ceph-mon[32580]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-23T18:30:34.444 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:34 smithi157 ceph-mon[32580]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T18:30:34.444 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:34 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:30:34.444 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:34 smithi157 ceph-mon[32580]: osd.5 [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653] boot 2024-04-23T18:30:34.444 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:34 smithi157 ceph-mon[32580]: osdmap e38: 6 total, 6 up, 6 in 2024-04-23T18:30:34.444 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:34 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:30:34.449 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T18:30:34.449 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T18:30:34.449 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-04-23T18:30:34.449 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-04-23T18:30:34.449 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0142315 s, 737 MB/s 2024-04-23T18:30:34.449 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-04-23T18:30:34.450 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:34 smithi132 ceph-mon[29178]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-23T18:30:34.450 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:34 smithi132 ceph-mon[29178]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T18:30:34.450 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:34 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:30:34.450 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:34 smithi132 ceph-mon[29178]: osd.5 [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653] boot 2024-04-23T18:30:34.450 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:34 smithi132 ceph-mon[29178]: osdmap e38: 6 total, 6 up, 6 in 2024-04-23T18:30:34.450 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:34 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:30:34.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:34 smithi148 ceph-mon[32311]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-23T18:30:34.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:34 smithi148 ceph-mon[32311]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T18:30:34.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:34 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:30:34.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:34 smithi148 ceph-mon[32311]: osd.5 [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653] boot 2024-04-23T18:30:34.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:34 smithi148 ceph-mon[32311]: osdmap e38: 6 total, 6 up, 6 in 2024-04-23T18:30:34.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:34 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:30:35.128 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi132:vg_nvme/lv_2 2024-04-23T18:30:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:35 smithi132 ceph-mon[29178]: purged_snaps scrub starts 2024-04-23T18:30:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:35 smithi132 ceph-mon[29178]: purged_snaps scrub ok 2024-04-23T18:30:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:35 smithi132 ceph-mon[29178]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:35 smithi132 ceph-mon[29178]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T18:30:35.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:35.369 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:30:35.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:35 smithi148 ceph-mon[32311]: purged_snaps scrub starts 2024-04-23T18:30:35.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:35 smithi148 ceph-mon[32311]: purged_snaps scrub ok 2024-04-23T18:30:35.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:35 smithi148 ceph-mon[32311]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:35.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:35.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:35.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:35.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:35.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:35.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:35.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:35.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:35 smithi148 ceph-mon[32311]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T18:30:35.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:35.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:35 smithi157 ceph-mon[32580]: purged_snaps scrub starts 2024-04-23T18:30:35.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:35 smithi157 ceph-mon[32580]: purged_snaps scrub ok 2024-04-23T18:30:35.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:35 smithi157 ceph-mon[32580]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:30:35.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:35.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:35.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:35.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:35.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:35.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:35.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:35.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:35 smithi157 ceph-mon[32580]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T18:30:35.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:36.431 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:36 smithi148 ceph-mon[32311]: Detected new or changed devices on smithi157 2024-04-23T18:30:36.431 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:36 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi157 to 8544M 2024-04-23T18:30:36.431 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:36 smithi148 ceph-mon[32311]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T18:30:36.508 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:36 smithi132 ceph-mon[29178]: Detected new or changed devices on smithi157 2024-04-23T18:30:36.508 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:36 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi157 to 8544M 2024-04-23T18:30:36.508 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:36 smithi132 ceph-mon[29178]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T18:30:36.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:36 smithi157 ceph-mon[32580]: Detected new or changed devices on smithi157 2024-04-23T18:30:36.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:36 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi157 to 8544M 2024-04-23T18:30:36.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:36 smithi157 ceph-mon[32580]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T18:30:37.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.089+0000 7faa09101700 1 -- 172.21.15.132:0/1182110241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa041019c0 msgr2=0x7faa04101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:37.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.089+0000 7faa09101700 1 --2- 172.21.15.132:0/1182110241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa041019c0 0x7faa04101de0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80099c0 tx=0x7fa9f8009cd0 comp rx=0 tx=0).stop 2024-04-23T18:30:37.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.089+0000 7faa09101700 1 -- 172.21.15.132:0/1182110241 shutdown_connections 2024-04-23T18:30:37.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.089+0000 7faa09101700 1 --2- 172.21.15.132:0/1182110241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04103560 0x7faa04109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:37.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.089+0000 7faa09101700 1 --2- 172.21.15.132:0/1182110241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa041019c0 0x7faa04101de0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:37.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.089+0000 7faa09101700 1 --2- 172.21.15.132:0/1182110241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa04102bc0 0x7faa04103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:37.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.089+0000 7faa09101700 1 -- 172.21.15.132:0/1182110241 >> 172.21.15.132:0/1182110241 conn(0x7faa040fcfa0 msgr2=0x7faa040ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:37.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.089+0000 7faa09101700 1 -- 172.21.15.132:0/1182110241 shutdown_connections 2024-04-23T18:30:37.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.089+0000 7faa09101700 1 -- 172.21.15.132:0/1182110241 wait complete. 2024-04-23T18:30:37.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.090+0000 7faa09101700 1 Processor -- start 2024-04-23T18:30:37.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.090+0000 7faa09101700 1 -- start start 2024-04-23T18:30:37.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.090+0000 7faa09101700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa041019c0 0x7faa0406bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:37.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.090+0000 7faa09101700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04102bc0 0x7faa0419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:37.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.090+0000 7faa09101700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04103560 0x7faa0419ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:37.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.090+0000 7faa09101700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7faa0410d450 con 0x7faa04102bc0 2024-04-23T18:30:37.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.090+0000 7faa09101700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faa0410d2d0 con 0x7faa041019c0 2024-04-23T18:30:37.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.090+0000 7faa09101700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faa0410d5d0 con 0x7faa04103560 2024-04-23T18:30:37.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa0259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04102bc0 0x7faa0419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:37.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa02d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa041019c0 0x7faa0406bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:37.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa0259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04102bc0 0x7faa0419d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41674/0 (socket says 172.21.15.132:41674) 2024-04-23T18:30:37.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa0259c700 1 -- 172.21.15.132:0/2644875487 learned_addr learned my addr 172.21.15.132:0/2644875487 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:30:37.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa0359e700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04103560 0x7faa0419ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:37.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa0259c700 1 -- 172.21.15.132:0/2644875487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04103560 msgr2=0x7faa0419ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:37.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa0259c700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04103560 0x7faa0419ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:37.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa0259c700 1 -- 172.21.15.132:0/2644875487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa041019c0 msgr2=0x7faa0406bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa0259c700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa041019c0 0x7faa0406bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa0259c700 1 -- 172.21.15.132:0/2644875487 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa041a0850 con 0x7faa04102bc0 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa02d9d700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa041019c0 0x7faa0406bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa0259c700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04102bc0 0x7faa0419d8f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400d980 tx=0x7fa9f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7fa9ebfff700 1 -- 172.21.15.132:0/2644875487 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4016070 con 0x7faa04102bc0 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa09101700 1 -- 172.21.15.132:0/2644875487 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa041a0b40 con 0x7faa04102bc0 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7fa9ebfff700 1 -- 172.21.15.132:0/2644875487 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f4004830 con 0x7faa04102bc0 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7fa9ebfff700 1 -- 172.21.15.132:0/2644875487 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4005540 con 0x7faa04102bc0 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.091+0000 7faa09101700 1 -- 172.21.15.132:0/2644875487 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa041a10d0 con 0x7faa04102bc0 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.092+0000 7faa09101700 1 -- 172.21.15.132:0/2644875487 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa04069730 con 0x7faa04102bc0 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.094+0000 7fa9ebfff700 1 -- 172.21.15.132:0/2644875487 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa9f4004220 con 0x7faa04102bc0 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.095+0000 7fa9ebfff700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa9ec0a3c40 0x7fa9ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.095+0000 7faa02d9d700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa9ec0a3c40 0x7fa9ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:37.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.095+0000 7fa9ebfff700 1 -- 172.21.15.132:0/2644875487 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fa9f40cf870 con 0x7faa04102bc0 2024-04-23T18:30:37.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.095+0000 7faa02d9d700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa9ec0a3c40 0x7fa9ec0a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80038c0 tx=0x7fa9f800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:37.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.098+0000 7fa9ebfff700 1 -- 172.21.15.132:0/2644875487 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9f401f020 con 0x7faa04102bc0 2024-04-23T18:30:37.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:37.308+0000 7faa09101700 1 -- 172.21.15.132:0/2644875487 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7faa04062e80 con 0x7fa9ec0a3c40 2024-04-23T18:30:37.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:37 smithi132 ceph-mon[29178]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:37.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:37 smithi148 ceph-mon[32311]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:37.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:37 smithi157 ceph-mon[32580]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:38.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:38 smithi148 ceph-mon[32311]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T18:30:38.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:30:38.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:30:38.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:38.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:38 smithi157 ceph-mon[32580]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T18:30:38.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:30:38.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:30:38.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:38.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:38 smithi132 ceph-mon[29178]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T18:30:38.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:30:38.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:30:38.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:39.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:39 smithi148 ceph-mon[32311]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:30:39.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:39 smithi148 ceph-mon[32311]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:39.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:39 smithi148 ceph-mon[32311]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T18:30:39.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:39 smithi157 ceph-mon[32580]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:30:39.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:39 smithi157 ceph-mon[32580]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:39.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:39 smithi157 ceph-mon[32580]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T18:30:39.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:39 smithi132 ceph-mon[29178]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:30:39.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:39 smithi132 ceph-mon[29178]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:39.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:39 smithi132 ceph-mon[29178]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T18:30:40.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:40 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/169374524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b1233e4-b273-4c3e-89a4-ecec0758d4fe"}]: dispatch 2024-04-23T18:30:40.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:40 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/169374524' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b1233e4-b273-4c3e-89a4-ecec0758d4fe"}]': finished 2024-04-23T18:30:40.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:40 smithi148 ceph-mon[32311]: osdmap e43: 7 total, 6 up, 7 in 2024-04-23T18:30:40.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:40 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:30:40.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:40 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/169374524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b1233e4-b273-4c3e-89a4-ecec0758d4fe"}]: dispatch 2024-04-23T18:30:40.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:40 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/169374524' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b1233e4-b273-4c3e-89a4-ecec0758d4fe"}]': finished 2024-04-23T18:30:40.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:40 smithi157 ceph-mon[32580]: osdmap e43: 7 total, 6 up, 7 in 2024-04-23T18:30:40.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:40 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:30:40.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:40 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/169374524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b1233e4-b273-4c3e-89a4-ecec0758d4fe"}]: dispatch 2024-04-23T18:30:40.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:40 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/169374524' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b1233e4-b273-4c3e-89a4-ecec0758d4fe"}]': finished 2024-04-23T18:30:40.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:40 smithi132 ceph-mon[29178]: osdmap e43: 7 total, 6 up, 7 in 2024-04-23T18:30:40.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:40 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:30:41.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:41 smithi148 ceph-mon[32311]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:41.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:41 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/4072012304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:30:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:41 smithi157 ceph-mon[32580]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:41 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/4072012304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:30:41.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:41 smithi132 ceph-mon[29178]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:41.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:41 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/4072012304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:30:43.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:43 smithi148 ceph-mon[32311]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-23T18:30:43.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:43 smithi157 ceph-mon[32580]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-23T18:30:43.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:43 smithi132 ceph-mon[29178]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-23T18:30:45.326 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:45 smithi132 ceph-mon[29178]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-23T18:30:45.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:45 smithi148 ceph-mon[32311]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-23T18:30:45.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:45 smithi157 ceph-mon[32580]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-23T18:30:47.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:47 smithi132 ceph-mon[29178]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:30:47.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:47 smithi148 ceph-mon[32311]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:30:47.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:47 smithi157 ceph-mon[32580]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:30:48.352 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:48 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T18:30:48.352 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:48 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:48.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:48 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T18:30:48.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:48 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:48.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:48 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T18:30:48.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:48 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:49.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:49 smithi132 ceph-mon[29178]: Deploying daemon osd.6 on smithi132 2024-04-23T18:30:49.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:49 smithi132 ceph-mon[29178]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:30:49.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:49 smithi148 ceph-mon[32311]: Deploying daemon osd.6 on smithi132 2024-04-23T18:30:49.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:49 smithi148 ceph-mon[32311]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:30:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:49 smithi157 ceph-mon[32580]: Deploying daemon osd.6 on smithi132 2024-04-23T18:30:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:49 smithi157 ceph-mon[32580]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:30:51.474 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:51 smithi132 ceph-mon[29178]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:30:51.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:51 smithi148 ceph-mon[32311]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:30:51.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:51 smithi157 ceph-mon[32580]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:30:53.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:53 smithi132 ceph-mon[29178]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:30:53.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:53 smithi148 ceph-mon[32311]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:30:53.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:53 smithi157 ceph-mon[32580]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:30:54.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:54.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:54.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:54.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:54.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:54.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:55.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:55 smithi132 ceph-mon[29178]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:55.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:55 smithi148 ceph-mon[32311]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:55 smithi157 ceph-mon[32580]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:57.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:57 smithi132 ceph-mon[29178]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:57.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:57.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:57.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:57.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:57.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:57.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:57 smithi132 ceph-mon[29178]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T18:30:57.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.373+0000 7fa9ebfff700 1 -- 172.21.15.132:0/2644875487 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faa04062e80 con 0x7fa9ec0a3c40 2024-04-23T18:30:57.366 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 6 on host 'smithi132' 2024-04-23T18:30:57.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.377+0000 7faa09101700 1 -- 172.21.15.132:0/2644875487 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa9ec0a3c40 msgr2=0x7fa9ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:57.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.377+0000 7faa09101700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa9ec0a3c40 0x7fa9ec0a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80038c0 tx=0x7fa9f800b540 comp rx=0 tx=0).stop 2024-04-23T18:30:57.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.377+0000 7faa09101700 1 -- 172.21.15.132:0/2644875487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04102bc0 msgr2=0x7faa0419d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:57.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.377+0000 7faa09101700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04102bc0 0x7faa0419d8f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400d980 tx=0x7fa9f400dc90 comp rx=0 tx=0).stop 2024-04-23T18:30:57.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.377+0000 7faa09101700 1 -- 172.21.15.132:0/2644875487 shutdown_connections 2024-04-23T18:30:57.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.377+0000 7faa09101700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa04103560 0x7faa0419ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:57.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.377+0000 7faa09101700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa041019c0 0x7faa0406bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:57.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.377+0000 7faa09101700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa9ec0a3c40 0x7fa9ec0a6100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:57.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.377+0000 7faa09101700 1 --2- 172.21.15.132:0/2644875487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa04102bc0 0x7faa0419d8f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:57.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.377+0000 7faa09101700 1 -- 172.21.15.132:0/2644875487 >> 172.21.15.132:0/2644875487 conn(0x7faa040fcfa0 msgr2=0x7faa04105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:57.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.377+0000 7faa09101700 1 -- 172.21.15.132:0/2644875487 shutdown_connections 2024-04-23T18:30:57.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:30:57.377+0000 7faa09101700 1 -- 172.21.15.132:0/2644875487 wait complete. 2024-04-23T18:30:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:57 smithi148 ceph-mon[32311]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:57.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:57.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:57.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:57.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:57.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:57.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:57 smithi148 ceph-mon[32311]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T18:30:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:57 smithi157 ceph-mon[32580]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:57 smithi157 ceph-mon[32580]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T18:30:58.037 DEBUG:teuthology.orchestra.run.smithi132:osd.6> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.6.service 2024-04-23T18:30:58.039 INFO:tasks.cephadm:Deploying osd.7 on smithi148 with /dev/vg_nvme/lv_2... 2024-04-23T18:30:58.040 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T18:30:58.293 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:30:58.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:58.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:58.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:58 smithi132 ceph-mon[29178]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T18:30:58.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:58 smithi132 ceph-mon[29178]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T18:30:58.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:58 smithi132 ceph-mon[29178]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-04-23T18:30:58.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:30:58.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:58.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:58 smithi132 ceph-mon[29178]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:58.657 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:58.657 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:58.657 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:58 smithi148 ceph-mon[32311]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T18:30:58.657 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:58 smithi148 ceph-mon[32311]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T18:30:58.657 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:58 smithi148 ceph-mon[32311]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-04-23T18:30:58.657 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:30:58.657 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:58.657 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:58 smithi148 ceph-mon[32311]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:58.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:58.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:30:58.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:58 smithi157 ceph-mon[32580]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T18:30:58.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:58 smithi157 ceph-mon[32580]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T18:30:58.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:58 smithi157 ceph-mon[32580]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-04-23T18:30:58.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:30:58.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:58.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:58 smithi157 ceph-mon[32580]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:30:59.112 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:30:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:30:58.779+0000 7f3b6762f700 -1 osd.6 0 waiting for initial osdmap 2024-04-23T18:30:59.112 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:30:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:30:58.791+0000 7f3b5f41a700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:30:59.610 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:59 smithi132 ceph-mon[29178]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-04-23T18:30:59.610 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:59 smithi132 ceph-mon[29178]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T18:30:59.610 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:30:59.610 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:30:59.610 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:30:59 smithi132 ceph-mon[29178]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' 2024-04-23T18:30:59.644 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:59 smithi148 ceph-mon[32311]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-04-23T18:30:59.644 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:59 smithi148 ceph-mon[32311]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T18:30:59.644 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:30:59.644 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:30:59.644 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:30:59 smithi148 ceph-mon[32311]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' 2024-04-23T18:30:59.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:59 smithi157 ceph-mon[32580]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-04-23T18:30:59.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:59 smithi157 ceph-mon[32580]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T18:30:59.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:30:59.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:30:59.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:30:59 smithi157 ceph-mon[32580]: from='osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226]' entity='osd.6' 2024-04-23T18:31:00.486 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:00 smithi148 ceph-mon[32311]: purged_snaps scrub starts 2024-04-23T18:31:00.486 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:00 smithi148 ceph-mon[32311]: purged_snaps scrub ok 2024-04-23T18:31:00.486 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:00 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:31:00.486 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:00 smithi148 ceph-mon[32311]: osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226] boot 2024-04-23T18:31:00.486 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:00 smithi148 ceph-mon[32311]: osdmap e46: 7 total, 7 up, 7 in 2024-04-23T18:31:00.486 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:00 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:31:00.487 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:00 smithi148 ceph-mon[32311]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:31:00.568 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T18:31:00.568 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T18:31:00.568 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-04-23T18:31:00.568 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-04-23T18:31:00.568 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133266 s, 787 MB/s 2024-04-23T18:31:00.568 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-04-23T18:31:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:00 smithi132 ceph-mon[29178]: purged_snaps scrub starts 2024-04-23T18:31:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:00 smithi132 ceph-mon[29178]: purged_snaps scrub ok 2024-04-23T18:31:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:00 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:31:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:00 smithi132 ceph-mon[29178]: osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226] boot 2024-04-23T18:31:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:00 smithi132 ceph-mon[29178]: osdmap e46: 7 total, 7 up, 7 in 2024-04-23T18:31:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:00 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:31:00.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:00 smithi132 ceph-mon[29178]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:31:00.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:00 smithi157 ceph-mon[32580]: purged_snaps scrub starts 2024-04-23T18:31:00.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:00 smithi157 ceph-mon[32580]: purged_snaps scrub ok 2024-04-23T18:31:00.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:00 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:31:00.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:00 smithi157 ceph-mon[32580]: osd.6 [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226] boot 2024-04-23T18:31:00.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:00 smithi157 ceph-mon[32580]: osdmap e46: 7 total, 7 up, 7 in 2024-04-23T18:31:00.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:00 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:31:00.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:00 smithi157 ceph-mon[32580]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:31:01.279 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_2 2024-04-23T18:31:01.522 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:31:01.971 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:01 smithi148 ceph-mon[32311]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T18:31:01.971 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:01 smithi148 ceph-mon[32311]: Detected new or changed devices on smithi132 2024-04-23T18:31:01.971 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:01.971 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:01.971 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:01.971 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:01.971 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:01.971 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:01 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi132 to 5696M 2024-04-23T18:31:01.971 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:01.972 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:01.972 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:01.972 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:02.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:01 smithi157 ceph-mon[32580]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T18:31:02.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:01 smithi157 ceph-mon[32580]: Detected new or changed devices on smithi132 2024-04-23T18:31:02.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:02.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:02.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:02.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:02.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:02.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:01 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi132 to 5696M 2024-04-23T18:31:02.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:02.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:02.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:02.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:02.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:01 smithi132 ceph-mon[29178]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T18:31:02.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:01 smithi132 ceph-mon[29178]: Detected new or changed devices on smithi132 2024-04-23T18:31:02.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:02.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:02.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:02.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:02.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:02.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:01 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi132 to 5696M 2024-04-23T18:31:02.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:02.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:02.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:02.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:02.987 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:02 smithi148 ceph-mon[32311]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:03.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:02 smithi157 ceph-mon[32580]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:03.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:02 smithi132 ceph-mon[29178]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:03.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.239+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4019092509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff198105e80 msgr2=0x7ff198108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:03.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.239+0000 7ff19ea3f700 1 --2- 172.21.15.148:0/4019092509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff198105e80 0x7ff198108270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009990 tx=0x7ff18c009ca0 comp rx=0 tx=0).stop 2024-04-23T18:31:03.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.239+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4019092509 shutdown_connections 2024-04-23T18:31:03.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.239+0000 7ff19ea3f700 1 --2- 172.21.15.148:0/4019092509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1981087b0 0x7ff19810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:03.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.239+0000 7ff19ea3f700 1 --2- 172.21.15.148:0/4019092509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff198103550 0x7ff198105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:03.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.239+0000 7ff19ea3f700 1 --2- 172.21.15.148:0/4019092509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff198105e80 0x7ff198108270 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:03.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.239+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4019092509 >> 172.21.15.148:0/4019092509 conn(0x7ff1980fcfa0 msgr2=0x7ff1980ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:03.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.239+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4019092509 shutdown_connections 2024-04-23T18:31:03.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.239+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4019092509 wait complete. 2024-04-23T18:31:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff19ea3f700 1 Processor -- start 2024-04-23T18:31:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff19ea3f700 1 -- start start 2024-04-23T18:31:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff19ea3f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff198103550 0x7ff198198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff19ea3f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff198105e80 0x7ff198199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff19ea3f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1981087b0 0x7ff1981a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff19ea3f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff19810e550 con 0x7ff198103550 2024-04-23T18:31:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff19ea3f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff19810e3d0 con 0x7ff198105e80 2024-04-23T18:31:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff19ea3f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff19810e6d0 con 0x7ff1981087b0 2024-04-23T18:31:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff1977fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff198105e80 0x7ff198199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff1977fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff198105e80 0x7ff198199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41016/0 (socket says 172.21.15.148:41016) 2024-04-23T18:31:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff1977fe700 1 -- 172.21.15.148:0/4091712760 learned_addr learned my addr 172.21.15.148:0/4091712760 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff197fff700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff198103550 0x7ff198198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff19cfdc700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1981087b0 0x7ff1981a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff1977fe700 1 -- 172.21.15.148:0/4091712760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1981087b0 msgr2=0x7ff1981a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff1977fe700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1981087b0 0x7ff1981a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff1977fe700 1 -- 172.21.15.148:0/4091712760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff198103550 msgr2=0x7ff198198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff1977fe700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff198103550 0x7ff198198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.240+0000 7ff1977fe700 1 -- 172.21.15.148:0/4091712760 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1981a0890 con 0x7ff198105e80 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.241+0000 7ff19cfdc700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1981087b0 0x7ff1981a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.241+0000 7ff1977fe700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff198105e80 0x7ff198199180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff18c006010 tx=0x7ff18c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.241+0000 7ff1957fa700 1 -- 172.21.15.148:0/4091712760 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c027070 con 0x7ff198105e80 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.241+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4091712760 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1981a0b20 con 0x7ff198105e80 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.241+0000 7ff1957fa700 1 -- 172.21.15.148:0/4091712760 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff18c004080 con 0x7ff198105e80 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.241+0000 7ff1957fa700 1 -- 172.21.15.148:0/4091712760 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c022e00 con 0x7ff198105e80 2024-04-23T18:31:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.241+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4091712760 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1981a0fd0 con 0x7ff198105e80 2024-04-23T18:31:03.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.242+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4091712760 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff198069730 con 0x7ff198105e80 2024-04-23T18:31:03.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.243+0000 7ff1957fa700 1 -- 172.21.15.148:0/4091712760 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff18c02a070 con 0x7ff198105e80 2024-04-23T18:31:03.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.244+0000 7ff1957fa700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff1800a3c40 0x7ff1800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:03.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.244+0000 7ff1957fa700 1 -- 172.21.15.148:0/4091712760 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7ff18c0d0910 con 0x7ff198105e80 2024-04-23T18:31:03.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.244+0000 7ff197fff700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff1800a3c40 0x7ff1800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:03.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.244+0000 7ff197fff700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff1800a3c40 0x7ff1800a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff19819a120 tx=0x7ff18800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:03.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.248+0000 7ff1957fa700 1 -- 172.21.15.148:0/4091712760 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff18c013070 con 0x7ff198105e80 2024-04-23T18:31:03.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:03.451+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4091712760 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff198062e80 con 0x7ff1800a3c40 2024-04-23T18:31:04.059 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:03 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:31:04.059 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:03 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:31:04.060 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:03 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:04.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:03 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:31:04.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:03 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:31:04.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:03 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:04.211 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:03 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:31:04.211 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:03 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:31:04.211 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:03 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:05.091 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:04 smithi148 ceph-mon[32311]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:31:05.091 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:04 smithi148 ceph-mon[32311]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:05.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:04 smithi132 ceph-mon[29178]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:31:05.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:04 smithi132 ceph-mon[29178]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:05.135 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:04 smithi157 ceph-mon[32580]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:31:05.135 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:04 smithi157 ceph-mon[32580]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:07 smithi132 ceph-mon[29178]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:07 smithi132 ceph-mon[29178]: from='client.? 172.21.15.148:0/1765211878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93425a43-d24f-4456-81d6-239c59996a22"}]: dispatch 2024-04-23T18:31:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:07 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93425a43-d24f-4456-81d6-239c59996a22"}]: dispatch 2024-04-23T18:31:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:07 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93425a43-d24f-4456-81d6-239c59996a22"}]': finished 2024-04-23T18:31:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:07 smithi132 ceph-mon[29178]: osdmap e48: 8 total, 7 up, 8 in 2024-04-23T18:31:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:07 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:07 smithi132 ceph-mon[29178]: from='client.? 172.21.15.148:0/1715482081' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:31:07.518 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:07 smithi148 ceph-mon[32311]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:07.518 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:07 smithi148 ceph-mon[32311]: from='client.? 172.21.15.148:0/1765211878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93425a43-d24f-4456-81d6-239c59996a22"}]: dispatch 2024-04-23T18:31:07.518 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:07 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93425a43-d24f-4456-81d6-239c59996a22"}]: dispatch 2024-04-23T18:31:07.518 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:07 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93425a43-d24f-4456-81d6-239c59996a22"}]': finished 2024-04-23T18:31:07.518 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:07 smithi148 ceph-mon[32311]: osdmap e48: 8 total, 7 up, 8 in 2024-04-23T18:31:07.519 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:07 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:07.519 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:07 smithi148 ceph-mon[32311]: from='client.? 172.21.15.148:0/1715482081' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:31:07.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:07 smithi157 ceph-mon[32580]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:07.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:07 smithi157 ceph-mon[32580]: from='client.? 172.21.15.148:0/1765211878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93425a43-d24f-4456-81d6-239c59996a22"}]: dispatch 2024-04-23T18:31:07.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:07 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93425a43-d24f-4456-81d6-239c59996a22"}]: dispatch 2024-04-23T18:31:07.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:07 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93425a43-d24f-4456-81d6-239c59996a22"}]': finished 2024-04-23T18:31:07.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:07 smithi157 ceph-mon[32580]: osdmap e48: 8 total, 7 up, 8 in 2024-04-23T18:31:07.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:07 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:07.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:07 smithi157 ceph-mon[32580]: from='client.? 172.21.15.148:0/1715482081' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:31:09.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:09 smithi148 ceph-mon[32311]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:09.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:09 smithi157 ceph-mon[32580]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:09.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:09 smithi132 ceph-mon[29178]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:11.489 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:11 smithi148 ceph-mon[32311]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:11.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:11 smithi157 ceph-mon[32580]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:11.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:11 smithi132 ceph-mon[29178]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:13.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:13 smithi148 ceph-mon[32311]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:13.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:13 smithi157 ceph-mon[32580]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:13.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:13 smithi132 ceph-mon[29178]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:14.465 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:14 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T18:31:14.465 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:14 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:14.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:14 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T18:31:14.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:14 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:14.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:14 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T18:31:14.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:14 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:15 smithi148 ceph-mon[32311]: Deploying daemon osd.7 on smithi148 2024-04-23T18:31:15.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:15 smithi148 ceph-mon[32311]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:15 smithi157 ceph-mon[32580]: Deploying daemon osd.7 on smithi148 2024-04-23T18:31:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:15 smithi157 ceph-mon[32580]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:15.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:15 smithi132 ceph-mon[29178]: Deploying daemon osd.7 on smithi148 2024-04-23T18:31:15.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:15 smithi132 ceph-mon[29178]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:17.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:17 smithi148 ceph-mon[32311]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:17.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:17 smithi157 ceph-mon[32580]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:17.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:17 smithi132 ceph-mon[29178]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:19.437 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:19 smithi148 ceph-mon[32311]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:19.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:19 smithi157 ceph-mon[32580]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:19.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:19 smithi132 ceph-mon[29178]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:21.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:21.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:21.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:21.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:20 smithi148 ceph-mon[32311]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:20 smithi157 ceph-mon[32580]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:20 smithi132 ceph-mon[29178]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:23.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:23 smithi148 ceph-mon[32311]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:23.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:23.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:23.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:23.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:23.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:23.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:23 smithi157 ceph-mon[32580]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:23.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:23.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:23.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:23.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:23.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:23.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:23 smithi132 ceph-mon[29178]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:23.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:23.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:23.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:23.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:23.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:23.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.671+0000 7ff1957fa700 1 -- 172.21.15.148:0/4091712760 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff198062e80 con 0x7ff1800a3c40 2024-04-23T18:31:23.663 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 7 on host 'smithi148' 2024-04-23T18:31:23.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.673+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4091712760 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff1800a3c40 msgr2=0x7ff1800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.673+0000 7ff19ea3f700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff1800a3c40 0x7ff1800a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff19819a120 tx=0x7ff18800b410 comp rx=0 tx=0).stop 2024-04-23T18:31:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.673+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4091712760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff198105e80 msgr2=0x7ff198199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.673+0000 7ff19ea3f700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff198105e80 0x7ff198199180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff18c006010 tx=0x7ff18c0047b0 comp rx=0 tx=0).stop 2024-04-23T18:31:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.673+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4091712760 shutdown_connections 2024-04-23T18:31:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.673+0000 7ff19ea3f700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1981087b0 0x7ff1981a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.673+0000 7ff19ea3f700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff1800a3c40 0x7ff1800a6100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.673+0000 7ff19ea3f700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff198103550 0x7ff198198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.673+0000 7ff19ea3f700 1 --2- 172.21.15.148:0/4091712760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff198105e80 0x7ff198199180 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.673+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4091712760 >> 172.21.15.148:0/4091712760 conn(0x7ff1980fcfa0 msgr2=0x7ff198103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.674+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4091712760 shutdown_connections 2024-04-23T18:31:23.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:31:23.674+0000 7ff19ea3f700 1 -- 172.21.15.148:0/4091712760 wait complete. 2024-04-23T18:31:24.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:24 smithi148 ceph-mon[32311]: from='osd.7 [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:31:24.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:24 smithi148 ceph-mon[32311]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:31:24.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:24 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:24.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:24 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:24.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:24 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:24.406 DEBUG:teuthology.orchestra.run.smithi148:osd.7> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.7.service 2024-04-23T18:31:24.408 INFO:tasks.cephadm:Deploying osd.8 on smithi157 with /dev/vg_nvme/lv_2... 2024-04-23T18:31:24.408 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T18:31:24.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:24 smithi157 ceph-mon[32580]: from='osd.7 [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:31:24.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:24 smithi157 ceph-mon[32580]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:31:24.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:24 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:24.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:24 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:24.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:24 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:24.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:24 smithi132 ceph-mon[29178]: from='osd.7 [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:31:24.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:24 smithi132 ceph-mon[29178]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:31:24.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:24 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:24.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:24 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:24.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:24 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:24.657 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:31:25.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:25 smithi148 ceph-mon[32311]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:25.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:25 smithi148 ceph-mon[32311]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T18:31:25.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:25 smithi148 ceph-mon[32311]: from='osd.7 [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:31:25.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:25 smithi148 ceph-mon[32311]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T18:31:25.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:25 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:25.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:25 smithi148 ceph-mon[32311]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:31:25.366 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:25 smithi157 ceph-mon[32580]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:25.366 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:25 smithi157 ceph-mon[32580]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T18:31:25.366 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:25 smithi157 ceph-mon[32580]: from='osd.7 [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:31:25.366 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:25 smithi157 ceph-mon[32580]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T18:31:25.366 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:25 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:25.366 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:25 smithi157 ceph-mon[32580]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:31:25.513 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:25 smithi132 ceph-mon[29178]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:25.513 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:25 smithi132 ceph-mon[29178]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T18:31:25.513 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:25 smithi132 ceph-mon[29178]: from='osd.7 [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:31:25.513 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:25 smithi132 ceph-mon[29178]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T18:31:25.513 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:25 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:25.514 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:25 smithi132 ceph-mon[29178]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:31:25.843 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:31:25 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:31:25.586+0000 7fbf7dc91700 -1 osd.7 0 waiting for initial osdmap 2024-04-23T18:31:25.843 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:31:25 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:31:25.594+0000 7fbf75a7c700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:31:26.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:26 smithi157 ceph-mon[32580]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-23T18:31:26.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:26 smithi157 ceph-mon[32580]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T18:31:26.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:26 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:26.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:26 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:26.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:26 smithi157 ceph-mon[32580]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T18:31:26.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:26 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:26.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:26 smithi157 ceph-mon[32580]: from='osd.7 ' entity='osd.7' 2024-04-23T18:31:26.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:26 smithi148 ceph-mon[32311]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-23T18:31:26.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:26 smithi148 ceph-mon[32311]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T18:31:26.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:26 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:26.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:26 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:26.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:26 smithi148 ceph-mon[32311]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T18:31:26.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:26 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:26.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:26 smithi148 ceph-mon[32311]: from='osd.7 ' entity='osd.7' 2024-04-23T18:31:26.568 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T18:31:26.568 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T18:31:26.568 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-04-23T18:31:26.569 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-04-23T18:31:26.569 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00852059 s, 1.2 GB/s 2024-04-23T18:31:26.569 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-04-23T18:31:26.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:26 smithi132 ceph-mon[29178]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-23T18:31:26.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:26 smithi132 ceph-mon[29178]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T18:31:26.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:26.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:26.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:26 smithi132 ceph-mon[29178]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T18:31:26.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:26.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:26 smithi132 ceph-mon[29178]: from='osd.7 ' entity='osd.7' 2024-04-23T18:31:27.187 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_2 2024-04-23T18:31:27.470 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:27 smithi157 ceph-mon[32580]: purged_snaps scrub starts 2024-04-23T18:31:27.470 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:27 smithi157 ceph-mon[32580]: purged_snaps scrub ok 2024-04-23T18:31:27.470 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:27 smithi157 ceph-mon[32580]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:27.470 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:27 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:27.470 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:27 smithi157 ceph-mon[32580]: osd.7 [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413] boot 2024-04-23T18:31:27.470 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:27 smithi157 ceph-mon[32580]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T18:31:27.470 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:27 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:27.476 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:31:27.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:27 smithi148 ceph-mon[32311]: purged_snaps scrub starts 2024-04-23T18:31:27.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:27 smithi148 ceph-mon[32311]: purged_snaps scrub ok 2024-04-23T18:31:27.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:27 smithi148 ceph-mon[32311]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:27.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:27 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:27.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:27 smithi148 ceph-mon[32311]: osd.7 [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413] boot 2024-04-23T18:31:27.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:27 smithi148 ceph-mon[32311]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T18:31:27.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:27 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:27.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:27 smithi132 ceph-mon[29178]: purged_snaps scrub starts 2024-04-23T18:31:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:27 smithi132 ceph-mon[29178]: purged_snaps scrub ok 2024-04-23T18:31:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:27 smithi132 ceph-mon[29178]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:31:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:27 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:27 smithi132 ceph-mon[29178]: osd.7 [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413] boot 2024-04-23T18:31:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:27 smithi132 ceph-mon[29178]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T18:31:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:27 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:31:28.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T18:31:28.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: Detected new or changed devices on smithi148 2024-04-23T18:31:28.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:28.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:28.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:28.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:28.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:28.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi148 to 5696M 2024-04-23T18:31:28.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:28.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:28.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:28.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:28.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:28 smithi157 ceph-mon[32580]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:28.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T18:31:28.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: Detected new or changed devices on smithi148 2024-04-23T18:31:28.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:28.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:28.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:28.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:28.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:28.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi148 to 5696M 2024-04-23T18:31:28.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:28.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:28.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:28.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:28.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:28 smithi148 ceph-mon[32311]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:28.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T18:31:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: Detected new or changed devices on smithi148 2024-04-23T18:31:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi148 to 5696M 2024-04-23T18:31:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:28.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:28 smithi132 ceph-mon[29178]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:29.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 -- 172.21.15.157:0/915725799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07d8104730 msgr2=0x7f07d810afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:29.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 --2- 172.21.15.157:0/915725799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07d8104730 0x7f07d810afe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07d000b0a0 tx=0x7f07d000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:31:29.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 -- 172.21.15.157:0/915725799 shutdown_connections 2024-04-23T18:31:29.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 --2- 172.21.15.157:0/915725799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07d8104730 0x7f07d810afe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:29.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 --2- 172.21.15.157:0/915725799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d8100db0 0x7f07d81011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:29.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 --2- 172.21.15.157:0/915725799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07d8103d90 0x7f07d81041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 -- 172.21.15.157:0/915725799 >> 172.21.15.157:0/915725799 conn(0x7f07d80fc710 msgr2=0x7f07d80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 -- 172.21.15.157:0/915725799 shutdown_connections 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 -- 172.21.15.157:0/915725799 wait complete. 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 Processor -- start 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 -- start start 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07d8100db0 0x7f07d81036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d8103d90 0x7f07d8101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07d8104730 0x7f07d8102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f07d810e580 con 0x7f07d8103d90 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f07d810e400 con 0x7f07d8100db0 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.096+0000 7f07e05f0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07d810e700 con 0x7f07d8104730 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07deb8d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07d8104730 0x7f07d8102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07deb8d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07d8104730 0x7f07d8102280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56212/0 (socket says 172.21.15.157:56212) 2024-04-23T18:31:29.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07deb8d700 1 -- 172.21.15.157:0/1326441871 learned_addr learned my addr 172.21.15.157:0/1326441871 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07ddb8b700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d8103d90 0x7f07d8101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07de38c700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07d8100db0 0x7f07d81036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07deb8d700 1 -- 172.21.15.157:0/1326441871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07d8100db0 msgr2=0x7f07d81036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07deb8d700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07d8100db0 0x7f07d81036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07deb8d700 1 -- 172.21.15.157:0/1326441871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d8103d90 msgr2=0x7f07d8101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07deb8d700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d8103d90 0x7f07d8101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07deb8d700 1 -- 172.21.15.157:0/1326441871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d8102b30 con 0x7f07d8104730 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07deb8d700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07d8104730 0x7f07d8102280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f07d00062a0 tx=0x7f07d0004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07cf7fe700 1 -- 172.21.15.157:0/1326441871 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0021070 con 0x7f07d8104730 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07e05f0700 1 -- 172.21.15.157:0/1326441871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d81a2c30 con 0x7f07d8104730 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07cf7fe700 1 -- 172.21.15.157:0/1326441871 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d0007c00 con 0x7f07d8104730 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07cf7fe700 1 -- 172.21.15.157:0/1326441871 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0004800 con 0x7f07d8104730 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.097+0000 7f07e05f0700 1 -- 172.21.15.157:0/1326441871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d81a3140 con 0x7f07d8104730 2024-04-23T18:31:29.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.098+0000 7f07e05f0700 1 -- 172.21.15.157:0/1326441871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d8069730 con 0x7f07d8104730 2024-04-23T18:31:29.094 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.099+0000 7f07cf7fe700 1 -- 172.21.15.157:0/1326441871 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f07d0003a00 con 0x7f07d8104730 2024-04-23T18:31:29.094 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.100+0000 7f07cf7fe700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f07c40a3c40 0x7f07c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:29.094 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.100+0000 7f07cf7fe700 1 -- 172.21.15.157:0/1326441871 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f07d000f020 con 0x7f07d8104730 2024-04-23T18:31:29.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.101+0000 7f07de38c700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f07c40a3c40 0x7f07c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:29.095 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.101+0000 7f07de38c700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f07c40a3c40 0x7f07c40a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f07d806af90 tx=0x7f07d400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:29.097 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.104+0000 7f07cf7fe700 1 -- 172.21.15.157:0/1326441871 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07d002a020 con 0x7f07d8104730 2024-04-23T18:31:29.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:29.311+0000 7f07e05f0700 1 -- 172.21.15.157:0/1326441871 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f07d8062f40 con 0x7f07c40a3c40 2024-04-23T18:31:29.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:29 smithi148 ceph-mon[32311]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T18:31:29.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:29 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:31:29.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:29 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:31:29.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:29 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:29.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:29 smithi157 ceph-mon[32580]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T18:31:29.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:29 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:31:29.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:29 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:31:29.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:29 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:29.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:29 smithi132 ceph-mon[29178]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T18:31:29.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:29 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:31:29.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:29 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:31:29.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:29 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:30.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:30 smithi148 ceph-mon[32311]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:31:30.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:30 smithi148 ceph-mon[32311]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T18:31:30.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:30 smithi148 ceph-mon[32311]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:30.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:30 smithi157 ceph-mon[32580]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:31:30.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:30 smithi157 ceph-mon[32580]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T18:31:30.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:30 smithi157 ceph-mon[32580]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:30.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:30 smithi132 ceph-mon[29178]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:31:30.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:30 smithi132 ceph-mon[29178]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T18:31:30.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:30 smithi132 ceph-mon[29178]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:31.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:31 smithi148 ceph-mon[32311]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T18:31:31.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:31 smithi157 ceph-mon[32580]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T18:31:31.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:31 smithi132 ceph-mon[29178]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T18:31:32.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:32 smithi148 ceph-mon[32311]: from='client.? 172.21.15.157:0/2556657141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9285b86-b921-4e03-af87-4fd814b866cc"}]: dispatch 2024-04-23T18:31:32.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:32 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9285b86-b921-4e03-af87-4fd814b866cc"}]: dispatch 2024-04-23T18:31:32.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:32 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9285b86-b921-4e03-af87-4fd814b866cc"}]': finished 2024-04-23T18:31:32.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:32 smithi148 ceph-mon[32311]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T18:31:32.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:32.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:32 smithi148 ceph-mon[32311]: pgmap v186: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:32.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:32 smithi157 ceph-mon[32580]: from='client.? 172.21.15.157:0/2556657141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9285b86-b921-4e03-af87-4fd814b866cc"}]: dispatch 2024-04-23T18:31:32.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:32 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9285b86-b921-4e03-af87-4fd814b866cc"}]: dispatch 2024-04-23T18:31:32.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:32 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9285b86-b921-4e03-af87-4fd814b866cc"}]': finished 2024-04-23T18:31:32.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:32 smithi157 ceph-mon[32580]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T18:31:32.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:32.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:32 smithi157 ceph-mon[32580]: pgmap v186: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:32.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:32 smithi132 ceph-mon[29178]: from='client.? 172.21.15.157:0/2556657141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9285b86-b921-4e03-af87-4fd814b866cc"}]: dispatch 2024-04-23T18:31:32.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:32 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9285b86-b921-4e03-af87-4fd814b866cc"}]: dispatch 2024-04-23T18:31:32.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:32 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9285b86-b921-4e03-af87-4fd814b866cc"}]': finished 2024-04-23T18:31:32.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:32 smithi132 ceph-mon[29178]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T18:31:32.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:32.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:32 smithi132 ceph-mon[29178]: pgmap v186: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:33.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:33 smithi148 ceph-mon[32311]: from='client.? 172.21.15.157:0/1301216512' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:31:33.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:33 smithi157 ceph-mon[32580]: from='client.? 172.21.15.157:0/1301216512' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:31:33.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:33 smithi132 ceph-mon[29178]: from='client.? 172.21.15.157:0/1301216512' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:31:34.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:34 smithi148 ceph-mon[32311]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:34.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:34 smithi157 ceph-mon[32580]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:34.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:34 smithi132 ceph-mon[29178]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:37.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:37 smithi157 ceph-mon[32580]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:37.468 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:37 smithi148 ceph-mon[32311]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:37.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:37 smithi132 ceph-mon[29178]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:39.376 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:39 smithi157 ceph-mon[32580]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:39.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:39 smithi148 ceph-mon[32311]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:39.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:39 smithi132 ceph-mon[29178]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:40.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:40 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T18:31:40.278 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:40 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:40.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:40 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T18:31:40.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:40 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:40.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:40 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T18:31:40.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:40 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:41.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:41 smithi157 ceph-mon[32580]: Deploying daemon osd.8 on smithi157 2024-04-23T18:31:41.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:41 smithi157 ceph-mon[32580]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:41.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:41 smithi148 ceph-mon[32311]: Deploying daemon osd.8 on smithi157 2024-04-23T18:31:41.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:41 smithi148 ceph-mon[32311]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:41.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:41 smithi132 ceph-mon[29178]: Deploying daemon osd.8 on smithi157 2024-04-23T18:31:41.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:41 smithi132 ceph-mon[29178]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:43.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:43 smithi157 ceph-mon[32580]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:43.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:43 smithi148 ceph-mon[32311]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:43.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:43 smithi132 ceph-mon[29178]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:45.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:45 smithi148 ceph-mon[32311]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:45.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:45 smithi157 ceph-mon[32580]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:45.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:45 smithi132 ceph-mon[29178]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:46.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:46.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:46.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:46.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:46.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:46.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:47.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:47 smithi157 ceph-mon[32580]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:47.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:47 smithi148 ceph-mon[32311]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:47.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:47 smithi132 ceph-mon[29178]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:49.119 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 8 on host 'smithi157' 2024-04-23T18:31:49.119 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.127+0000 7f07cf7fe700 1 -- 172.21.15.157:0/1326441871 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f07d8062f40 con 0x7f07c40a3c40 2024-04-23T18:31:49.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.129+0000 7f07e05f0700 1 -- 172.21.15.157:0/1326441871 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f07c40a3c40 msgr2=0x7f07c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:49.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.129+0000 7f07e05f0700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f07c40a3c40 0x7f07c40a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f07d806af90 tx=0x7f07d400a3b0 comp rx=0 tx=0).stop 2024-04-23T18:31:49.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.129+0000 7f07e05f0700 1 -- 172.21.15.157:0/1326441871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07d8104730 msgr2=0x7f07d8102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:49.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.129+0000 7f07e05f0700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07d8104730 0x7f07d8102280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f07d00062a0 tx=0x7f07d0004e40 comp rx=0 tx=0).stop 2024-04-23T18:31:49.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.129+0000 7f07e05f0700 1 -- 172.21.15.157:0/1326441871 shutdown_connections 2024-04-23T18:31:49.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.129+0000 7f07e05f0700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07d8104730 0x7f07d8102280 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:49.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.129+0000 7f07e05f0700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07d8100db0 0x7f07d81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:49.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.129+0000 7f07e05f0700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f07c40a3c40 0x7f07c40a6100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:49.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.129+0000 7f07e05f0700 1 --2- 172.21.15.157:0/1326441871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d8103d90 0x7f07d8101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:49.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.129+0000 7f07e05f0700 1 -- 172.21.15.157:0/1326441871 >> 172.21.15.157:0/1326441871 conn(0x7f07d80fc710 msgr2=0x7f07d8107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:49.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.129+0000 7f07e05f0700 1 -- 172.21.15.157:0/1326441871 shutdown_connections 2024-04-23T18:31:49.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:31:49.129+0000 7f07e05f0700 1 -- 172.21.15.157:0/1326441871 wait complete. 2024-04-23T18:31:49.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:49 smithi157 ceph-mon[32580]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:49.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:49.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:49.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:49 smithi157 ceph-mon[32580]: from='osd.8 [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:31:49.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:49 smithi157 ceph-mon[32580]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:31:49.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:49.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:49 smithi148 ceph-mon[32311]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:49 smithi148 ceph-mon[32311]: from='osd.8 [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:31:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:49 smithi148 ceph-mon[32311]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:31:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:49.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:49 smithi132 ceph-mon[29178]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:49.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:49.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:49.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:49 smithi132 ceph-mon[29178]: from='osd.8 [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:31:49.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:49 smithi132 ceph-mon[29178]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:31:49.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:49.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:49.785 DEBUG:teuthology.orchestra.run.smithi157:osd.8> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.8.service 2024-04-23T18:31:49.787 INFO:tasks.cephadm:Deploying osd.9 on smithi132 with /dev/vg_nvme/lv_1... 2024-04-23T18:31:49.788 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T18:31:50.037 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:31:51.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:51 smithi157 ceph-mon[32580]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T18:31:51.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:51 smithi157 ceph-mon[32580]: from='osd.8 [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:31:51.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:51 smithi157 ceph-mon[32580]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T18:31:51.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:51 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:51.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:51 smithi157 ceph-mon[32580]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:31:51.193 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:51 smithi157 ceph-mon[32580]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:51.194 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:31:51 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:31:51.090+0000 7f53d5725700 -1 osd.8 0 waiting for initial osdmap 2024-04-23T18:31:51.194 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:31:51 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:31:51.100+0000 7f53d1518700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:31:51.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:51 smithi132 ceph-mon[29178]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T18:31:51.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:51 smithi132 ceph-mon[29178]: from='osd.8 [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:31:51.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:51 smithi132 ceph-mon[29178]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T18:31:51.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:51 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:51.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:51 smithi132 ceph-mon[29178]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:31:51.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:51 smithi132 ceph-mon[29178]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:51.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:51 smithi148 ceph-mon[32311]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T18:31:51.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:51 smithi148 ceph-mon[32311]: from='osd.8 [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:31:51.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:51 smithi148 ceph-mon[32311]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T18:31:51.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:51 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:51.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:51 smithi148 ceph-mon[32311]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:31:51.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:51 smithi148 ceph-mon[32311]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:52.144 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T18:31:52.145 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T18:31:52.145 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-04-23T18:31:52.145 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-04-23T18:31:52.145 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115847 s, 905 MB/s 2024-04-23T18:31:52.145 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-04-23T18:31:52.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:52 smithi132 ceph-mon[29178]: purged_snaps scrub starts 2024-04-23T18:31:52.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:52 smithi132 ceph-mon[29178]: purged_snaps scrub ok 2024-04-23T18:31:52.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:52 smithi132 ceph-mon[29178]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-23T18:31:52.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:52 smithi132 ceph-mon[29178]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T18:31:52.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:52 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:52.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:52 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:52.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:52 smithi148 ceph-mon[32311]: purged_snaps scrub starts 2024-04-23T18:31:52.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:52 smithi148 ceph-mon[32311]: purged_snaps scrub ok 2024-04-23T18:31:52.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:52 smithi148 ceph-mon[32311]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-23T18:31:52.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:52 smithi148 ceph-mon[32311]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T18:31:52.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:52 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:52.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:52 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:52.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:52 smithi157 ceph-mon[32580]: purged_snaps scrub starts 2024-04-23T18:31:52.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:52 smithi157 ceph-mon[32580]: purged_snaps scrub ok 2024-04-23T18:31:52.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:52 smithi157 ceph-mon[32580]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-23T18:31:52.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:52 smithi157 ceph-mon[32580]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T18:31:52.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:52 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:52.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:52 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:52.816 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi132:vg_nvme/lv_1 2024-04-23T18:31:53.065 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:31:53.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:53.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: osd.8 [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159] boot 2024-04-23T18:31:53.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T18:31:53.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:53.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:53.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:53.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:53.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:53.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:53.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:53.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:53.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:53.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:53.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:53 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:53.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: osd.8 [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159] boot 2024-04-23T18:31:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T18:31:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:53.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:53.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:53.363 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:53 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:53.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: osd.8 [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159] boot 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:53.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:53 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:31:54.344 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:54 smithi132 ceph-mon[29178]: Detected new or changed devices on smithi157 2024-04-23T18:31:54.344 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:54 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi157 to 5696M 2024-04-23T18:31:54.344 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:54 smithi132 ceph-mon[29178]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T18:31:54.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:54 smithi148 ceph-mon[32311]: Detected new or changed devices on smithi157 2024-04-23T18:31:54.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:54 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi157 to 5696M 2024-04-23T18:31:54.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:54 smithi148 ceph-mon[32311]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T18:31:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:54 smithi157 ceph-mon[32580]: Detected new or changed devices on smithi157 2024-04-23T18:31:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:54 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi157 to 5696M 2024-04-23T18:31:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:54 smithi157 ceph-mon[32580]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T18:31:54.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.802+0000 7f473f742700 1 -- 172.21.15.132:0/3770655578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4738108830 msgr2=0x7f473810ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:54.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.802+0000 7f473f742700 1 --2- 172.21.15.132:0/3770655578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4738108830 0x7f473810ac20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f473000b0a0 tx=0x7f473000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.802+0000 7f473f742700 1 -- 172.21.15.132:0/3770655578 shutdown_connections 2024-04-23T18:31:54.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.802+0000 7f473f742700 1 --2- 172.21.15.132:0/3770655578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4738108830 0x7f473810ac20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.802+0000 7f473f742700 1 --2- 172.21.15.132:0/3770655578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47381035d0 0x7f47381059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.802+0000 7f473f742700 1 --2- 172.21.15.132:0/3770655578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4738105f00 0x7f47381082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.802+0000 7f473f742700 1 -- 172.21.15.132:0/3770655578 >> 172.21.15.132:0/3770655578 conn(0x7f47380fcf40 msgr2=0x7f47380ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.803+0000 7f473f742700 1 -- 172.21.15.132:0/3770655578 shutdown_connections 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.803+0000 7f473f742700 1 -- 172.21.15.132:0/3770655578 wait complete. 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.803+0000 7f473f742700 1 Processor -- start 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.803+0000 7f473f742700 1 -- start start 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.803+0000 7f473f742700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47381035d0 0x7f4738198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.803+0000 7f473f742700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4738105f00 0x7f4738199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.803+0000 7f473f742700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4738108830 0x7f47381a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.803+0000 7f473f742700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f473810e600 con 0x7f4738105f00 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.803+0000 7f473f742700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f473810e480 con 0x7f4738108830 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.803+0000 7f473f742700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f473810e780 con 0x7f47381035d0 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473ccdd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4738105f00 0x7f4738199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:54.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473ccdd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4738105f00 0x7f4738199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50666/0 (socket says 172.21.15.132:50666) 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473ccdd700 1 -- 172.21.15.132:0/3461640535 learned_addr learned my addr 172.21.15.132:0/3461640535 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473d4de700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47381035d0 0x7f4738198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473dcdf700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4738108830 0x7f47381a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473ccdd700 1 -- 172.21.15.132:0/3461640535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47381035d0 msgr2=0x7f4738198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473ccdd700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47381035d0 0x7f4738198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473ccdd700 1 -- 172.21.15.132:0/3461640535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4738108830 msgr2=0x7f47381a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473ccdd700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4738108830 0x7f47381a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473ccdd700 1 -- 172.21.15.132:0/3461640535 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47381a0970 con 0x7f4738105f00 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473dcdf700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4738108830 0x7f47381a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473ccdd700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4738105f00 0x7f4738199260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f472800b530 tx=0x7f472800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f472e7fc700 1 -- 172.21.15.132:0/3461640535 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728015020 con 0x7f4738105f00 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473f742700 1 -- 172.21.15.132:0/3461640535 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47381a0c60 con 0x7f4738105f00 2024-04-23T18:31:54.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f472e7fc700 1 -- 172.21.15.132:0/3461640535 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4728011ce0 con 0x7f4738105f00 2024-04-23T18:31:54.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f472e7fc700 1 -- 172.21.15.132:0/3461640535 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728010d20 con 0x7f4738105f00 2024-04-23T18:31:54.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.804+0000 7f473f742700 1 -- 172.21.15.132:0/3461640535 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47381a11a0 con 0x7f4738105f00 2024-04-23T18:31:54.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.805+0000 7f473f742700 1 -- 172.21.15.132:0/3461640535 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4738069730 con 0x7f4738105f00 2024-04-23T18:31:54.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.806+0000 7f472e7fc700 1 -- 172.21.15.132:0/3461640535 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4728018020 con 0x7f4738105f00 2024-04-23T18:31:54.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.807+0000 7f472e7fc700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f47240a3c40 0x7f47240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:54.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.807+0000 7f473d4de700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f47240a3c40 0x7f47240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:54.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.807+0000 7f472e7fc700 1 -- 172.21.15.132:0/3461640535 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7f47280cf820 con 0x7f4738105f00 2024-04-23T18:31:54.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.807+0000 7f473d4de700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f47240a3c40 0x7f47240a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4734006fd0 tx=0x7f4734008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:54.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:54.811+0000 7f472e7fc700 1 -- 172.21.15.132:0/3461640535 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f472801f020 con 0x7f4738105f00 2024-04-23T18:31:55.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:31:55.019+0000 7f473f742700 1 -- 172.21.15.132:0/3461640535 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4738062e80 con 0x7f47240a3c40 2024-04-23T18:31:55.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:55 smithi132 ceph-mon[29178]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:31:55.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:55 smithi132 ceph-mon[29178]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T18:31:55.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:31:55.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:31:55.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:55 smithi148 ceph-mon[32311]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:31:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:55 smithi148 ceph-mon[32311]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T18:31:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:31:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:31:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:55 smithi157 ceph-mon[32580]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:31:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:55 smithi157 ceph-mon[32580]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T18:31:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:31:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:31:55.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:56.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:56 smithi132 ceph-mon[29178]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:31:56.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:56 smithi132 ceph-mon[29178]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T18:31:56.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:56 smithi148 ceph-mon[32311]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:31:56.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:56 smithi148 ceph-mon[32311]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T18:31:56.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:56 smithi157 ceph-mon[32580]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:31:56.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:56 smithi157 ceph-mon[32580]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T18:31:57.174 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:57 smithi132 ceph-mon[29178]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:31:57.175 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:57 smithi132 ceph-mon[29178]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T18:31:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:57 smithi148 ceph-mon[32311]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:31:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:57 smithi148 ceph-mon[32311]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T18:31:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:57 smithi157 ceph-mon[32580]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:31:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:57 smithi157 ceph-mon[32580]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T18:31:58.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:58 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2598871190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90c3d0c2-e221-41ac-8da7-b272837f6283"}]: dispatch 2024-04-23T18:31:58.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:58 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2598871190' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90c3d0c2-e221-41ac-8da7-b272837f6283"}]': finished 2024-04-23T18:31:58.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:58 smithi132 ceph-mon[29178]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T18:31:58.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:31:58.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:58 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2598871190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90c3d0c2-e221-41ac-8da7-b272837f6283"}]: dispatch 2024-04-23T18:31:58.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:58 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2598871190' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90c3d0c2-e221-41ac-8da7-b272837f6283"}]': finished 2024-04-23T18:31:58.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:58 smithi148 ceph-mon[32311]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T18:31:58.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:31:58.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:58 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2598871190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90c3d0c2-e221-41ac-8da7-b272837f6283"}]: dispatch 2024-04-23T18:31:58.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:58 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2598871190' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90c3d0c2-e221-41ac-8da7-b272837f6283"}]': finished 2024-04-23T18:31:58.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:58 smithi157 ceph-mon[32580]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T18:31:58.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:31:59.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:59 smithi148 ceph-mon[32311]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:31:59.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:31:59 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2543290266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:31:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:59 smithi157 ceph-mon[32580]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:31:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:31:59 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2543290266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:31:59.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:59 smithi132 ceph-mon[29178]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:31:59.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:31:59 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2543290266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:32:01.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:01 smithi148 ceph-mon[32311]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:32:01.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:01 smithi157 ceph-mon[32580]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:32:01.558 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:01 smithi132 ceph-mon[29178]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:32:03.274 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:03 smithi132 ceph-mon[29178]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-23T18:32:03.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:03 smithi148 ceph-mon[32311]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-23T18:32:03.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:03 smithi157 ceph-mon[32580]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-23T18:32:05.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:05 smithi132 ceph-mon[29178]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:32:05.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:05 smithi148 ceph-mon[32311]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:32:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:05 smithi157 ceph-mon[32580]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:32:06.448 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:06 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T18:32:06.449 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:06 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:06.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:06 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T18:32:06.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:06 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:06.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:06 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T18:32:06.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:06 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:07.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:07 smithi148 ceph-mon[32311]: Deploying daemon osd.9 on smithi132 2024-04-23T18:32:07.416 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:07 smithi148 ceph-mon[32311]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:32:07.503 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:07 smithi132 ceph-mon[29178]: Deploying daemon osd.9 on smithi132 2024-04-23T18:32:07.503 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:07 smithi132 ceph-mon[29178]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:32:07.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:07 smithi157 ceph-mon[32580]: Deploying daemon osd.9 on smithi132 2024-04-23T18:32:07.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:07 smithi157 ceph-mon[32580]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:32:09.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:09 smithi132 ceph-mon[29178]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:09.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:09 smithi148 ceph-mon[32311]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:09.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:09 smithi157 ceph-mon[32580]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:11.349 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:11 smithi132 ceph-mon[29178]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:11.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:11 smithi148 ceph-mon[32311]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:11.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:11 smithi157 ceph-mon[32580]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:12.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:12.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:12.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:12.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:12 smithi132 ceph-mon[29178]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:13.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:13.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:13.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:13.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:12 smithi148 ceph-mon[32311]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:13.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:13.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:13.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:13.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:12 smithi157 ceph-mon[32580]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:15.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:15 smithi132 ceph-mon[29178]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:15.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.477+0000 7f472e7fc700 1 -- 172.21.15.132:0/3461640535 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4738062e80 con 0x7f47240a3c40 2024-04-23T18:32:15.469 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 9 on host 'smithi132' 2024-04-23T18:32:15.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.480+0000 7f473f742700 1 -- 172.21.15.132:0/3461640535 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f47240a3c40 msgr2=0x7f47240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:15.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.480+0000 7f473f742700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f47240a3c40 0x7f47240a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4734006fd0 tx=0x7f4734008040 comp rx=0 tx=0).stop 2024-04-23T18:32:15.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.480+0000 7f473f742700 1 -- 172.21.15.132:0/3461640535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4738105f00 msgr2=0x7f4738199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:15.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.480+0000 7f473f742700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4738105f00 0x7f4738199260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f472800b530 tx=0x7f472800b8f0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.480+0000 7f473f742700 1 -- 172.21.15.132:0/3461640535 shutdown_connections 2024-04-23T18:32:15.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.480+0000 7f473f742700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4738108830 0x7f47381a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.480+0000 7f473f742700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47381035d0 0x7f4738198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.480+0000 7f473f742700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f47240a3c40 0x7f47240a6100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.480+0000 7f473f742700 1 --2- 172.21.15.132:0/3461640535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4738105f00 0x7f4738199260 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.480+0000 7f473f742700 1 -- 172.21.15.132:0/3461640535 >> 172.21.15.132:0/3461640535 conn(0x7f47380fcf40 msgr2=0x7f4738104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:15.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.480+0000 7f473f742700 1 -- 172.21.15.132:0/3461640535 shutdown_connections 2024-04-23T18:32:15.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:32:15.480+0000 7f473f742700 1 -- 172.21.15.132:0/3461640535 wait complete. 2024-04-23T18:32:15.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:15 smithi148 ceph-mon[32311]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:15.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:15 smithi157 ceph-mon[32580]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:16.188 DEBUG:teuthology.orchestra.run.smithi132:osd.9> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.9.service 2024-04-23T18:32:16.191 INFO:tasks.cephadm:Deploying osd.10 on smithi148 with /dev/vg_nvme/lv_1... 2024-04-23T18:32:16.191 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T18:32:16.441 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:32:16.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:16 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:16.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:16 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:16.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:16 smithi148 ceph-mon[32311]: from='osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T18:32:16.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:16 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:16.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:16 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:16.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:16 smithi148 ceph-mon[32311]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:16.763 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:16 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:16.763 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:16 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:16.763 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:16 smithi132 ceph-mon[29178]: from='osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T18:32:16.763 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:16 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:16.764 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:16 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:16.764 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:16 smithi132 ceph-mon[29178]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:16.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:16 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:16.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:16 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:16.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:16 smithi157 ceph-mon[32580]: from='osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T18:32:16.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:16 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:16.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:16 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:16.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:16 smithi157 ceph-mon[32580]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:17.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:17 smithi132 ceph-mon[29178]: from='osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T18:32:17.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:17 smithi132 ceph-mon[29178]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T18:32:17.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:17 smithi132 ceph-mon[29178]: from='osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-04-23T18:32:17.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:17 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:17.706 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:32:17 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:32:17.450+0000 7f91ed518700 -1 osd.9 0 waiting for initial osdmap 2024-04-23T18:32:17.706 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:32:17 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:32:17.465+0000 7f91e930b700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:32:17.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:17 smithi148 ceph-mon[32311]: from='osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T18:32:17.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:17 smithi148 ceph-mon[32311]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T18:32:17.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:17 smithi148 ceph-mon[32311]: from='osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-04-23T18:32:17.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:17 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:17.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:17 smithi157 ceph-mon[32580]: from='osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T18:32:17.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:17 smithi157 ceph-mon[32580]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T18:32:17.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:17 smithi157 ceph-mon[32580]: from='osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-04-23T18:32:17.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:17 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:18.521 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T18:32:18.521 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T18:32:18.521 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-04-23T18:32:18.521 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-04-23T18:32:18.522 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113735 s, 922 MB/s 2024-04-23T18:32:18.522 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-04-23T18:32:18.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:18 smithi148 ceph-mon[32311]: from='osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-04-23T18:32:18.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:18 smithi148 ceph-mon[32311]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T18:32:18.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:18 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:18.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:18 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:18.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:18 smithi148 ceph-mon[32311]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:18.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:18 smithi157 ceph-mon[32580]: from='osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-04-23T18:32:18.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:18 smithi157 ceph-mon[32580]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T18:32:18.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:18 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:18.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:18 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:18.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:18 smithi157 ceph-mon[32580]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:18.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:18 smithi132 ceph-mon[29178]: from='osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-04-23T18:32:18.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:18 smithi132 ceph-mon[29178]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T18:32:18.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:18 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:18.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:18 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:18.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:18 smithi132 ceph-mon[29178]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:32:19.243 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_1 2024-04-23T18:32:19.482 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:32:19.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:19 smithi148 ceph-mon[32311]: purged_snaps scrub starts 2024-04-23T18:32:19.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:19 smithi148 ceph-mon[32311]: purged_snaps scrub ok 2024-04-23T18:32:19.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:19 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:19.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:19 smithi148 ceph-mon[32311]: osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445] boot 2024-04-23T18:32:19.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:19 smithi148 ceph-mon[32311]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T18:32:19.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:19 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:19.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:19 smithi157 ceph-mon[32580]: purged_snaps scrub starts 2024-04-23T18:32:19.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:19 smithi157 ceph-mon[32580]: purged_snaps scrub ok 2024-04-23T18:32:19.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:19 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:19.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:19 smithi157 ceph-mon[32580]: osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445] boot 2024-04-23T18:32:19.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:19 smithi157 ceph-mon[32580]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T18:32:19.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:19 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:19.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:19 smithi132 ceph-mon[29178]: purged_snaps scrub starts 2024-04-23T18:32:19.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:19 smithi132 ceph-mon[29178]: purged_snaps scrub ok 2024-04-23T18:32:19.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:19 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:19.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:19 smithi132 ceph-mon[29178]: osd.9 [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445] boot 2024-04-23T18:32:19.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:19 smithi132 ceph-mon[29178]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T18:32:19.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:19 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:32:20.747 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: Detected new or changed devices on smithi132 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi132 to 4272M 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:20.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:20.749 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:20.749 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.749 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:20 smithi148 ceph-mon[32311]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T18:32:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: Detected new or changed devices on smithi132 2024-04-23T18:32:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:20.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi132 to 4272M 2024-04-23T18:32:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:20 smithi157 ceph-mon[32580]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:20.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T18:32:20.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: Detected new or changed devices on smithi132 2024-04-23T18:32:20.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:20.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:20.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:20.863 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:20.863 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi132 to 4272M 2024-04-23T18:32:20.863 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.863 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:20.863 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:20.863 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.863 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:20.863 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:20.863 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:20.863 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:20.863 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:20 smithi132 ceph-mon[29178]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:21.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.287+0000 7f6185f08700 1 -- 172.21.15.148:0/431608786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61801019a0 msgr2=0x7f6180101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:21.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.287+0000 7f6185f08700 1 --2- 172.21.15.148:0/431608786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61801019a0 0x7f6180101dc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f61700099c0 tx=0x7f6170009cd0 comp rx=0 tx=0).stop 2024-04-23T18:32:21.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.287+0000 7f6185f08700 1 -- 172.21.15.148:0/431608786 shutdown_connections 2024-04-23T18:32:21.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.287+0000 7f6185f08700 1 --2- 172.21.15.148:0/431608786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6180103540 0x7f6180109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:21.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.287+0000 7f6185f08700 1 --2- 172.21.15.148:0/431608786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61801019a0 0x7f6180101dc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:21.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.287+0000 7f6185f08700 1 --2- 172.21.15.148:0/431608786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6180102ba0 0x7f6180103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:21.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.287+0000 7f6185f08700 1 -- 172.21.15.148:0/431608786 >> 172.21.15.148:0/431608786 conn(0x7f61800fcf40 msgr2=0x7f61800ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:21.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.287+0000 7f6185f08700 1 -- 172.21.15.148:0/431608786 shutdown_connections 2024-04-23T18:32:21.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.287+0000 7f6185f08700 1 -- 172.21.15.148:0/431608786 wait complete. 2024-04-23T18:32:21.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f6185f08700 1 Processor -- start 2024-04-23T18:32:21.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f6185f08700 1 -- start start 2024-04-23T18:32:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f6185f08700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61801019a0 0x7f6180198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f6185f08700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6180102ba0 0x7f61801991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f6185f08700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6180103540 0x7f61801a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f6185f08700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f618010d430 con 0x7f61801019a0 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f6185f08700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f618010d2b0 con 0x7f6180103540 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f6185f08700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f618010d5b0 con 0x7f6180102ba0 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f617f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61801019a0 0x7f6180198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f617f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61801019a0 0x7f6180198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.148:45620/0 (socket says 172.21.15.148:45620) 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f617f7fe700 1 -- 172.21.15.148:0/217537558 learned_addr learned my addr 172.21.15.148:0/217537558 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f617ffff700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6180103540 0x7f61801a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.288+0000 7f6176ffd700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6180102ba0 0x7f61801991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.289+0000 7f617ffff700 1 -- 172.21.15.148:0/217537558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6180102ba0 msgr2=0x7f61801991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.289+0000 7f617ffff700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6180102ba0 0x7f61801991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.289+0000 7f617ffff700 1 -- 172.21.15.148:0/217537558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61801019a0 msgr2=0x7f6180198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.289+0000 7f617ffff700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61801019a0 0x7f6180198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.289+0000 7f617ffff700 1 -- 172.21.15.148:0/217537558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61801a08f0 con 0x7f6180103540 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.289+0000 7f617ffff700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6180103540 0x7f61801a0220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6178007f10 tx=0x7f617800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.289+0000 7f617d7fa700 1 -- 172.21.15.148:0/217537558 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178004db0 con 0x7f6180103540 2024-04-23T18:32:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.289+0000 7f617d7fa700 1 -- 172.21.15.148:0/217537558 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6178003a00 con 0x7f6180103540 2024-04-23T18:32:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.289+0000 7f6185f08700 1 -- 172.21.15.148:0/217537558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61801a0be0 con 0x7f6180103540 2024-04-23T18:32:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.289+0000 7f617d7fa700 1 -- 172.21.15.148:0/217537558 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178004580 con 0x7f6180103540 2024-04-23T18:32:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.289+0000 7f6185f08700 1 -- 172.21.15.148:0/217537558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61801a1120 con 0x7f6180103540 2024-04-23T18:32:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.290+0000 7f6185f08700 1 -- 172.21.15.148:0/217537558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6180069760 con 0x7f6180103540 2024-04-23T18:32:21.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.291+0000 7f617d7fa700 1 -- 172.21.15.148:0/217537558 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f617801c020 con 0x7f6180103540 2024-04-23T18:32:21.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.292+0000 7f617d7fa700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f616c0a3cc0 0x7f616c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:21.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.292+0000 7f617d7fa700 1 -- 172.21.15.148:0/217537558 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f61780cfd20 con 0x7f6180103540 2024-04-23T18:32:21.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.292+0000 7f617f7fe700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f616c0a3cc0 0x7f616c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:21.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.293+0000 7f617f7fe700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f616c0a3cc0 0x7f616c0a6180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6170000c00 tx=0x7f61700036c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.296+0000 7f617d7fa700 1 -- 172.21.15.148:0/217537558 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6180069760 con 0x7f6180103540 2024-04-23T18:32:21.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:21.501+0000 7f6185f08700 1 -- 172.21.15.148:0/217537558 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f618006ae30 con 0x7f616c0a3cc0 2024-04-23T18:32:21.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:21 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:32:21.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:21 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:32:21.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:21 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:22.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:21 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:32:22.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:21 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:32:22.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:21 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:22.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:21 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:32:22.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:21 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:32:22.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:21 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:23.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:22 smithi148 ceph-mon[32311]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:32:23.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:22 smithi148 ceph-mon[32311]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:23.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:22 smithi157 ceph-mon[32580]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:32:23.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:22 smithi157 ceph-mon[32580]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:23.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:22 smithi132 ceph-mon[29178]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:32:23.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:22 smithi132 ceph-mon[29178]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:25 smithi148 ceph-mon[32311]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:25 smithi148 ceph-mon[32311]: from='client.? 172.21.15.148:0/4130567983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e548975f-da5d-4394-a5c6-e658a6c89046"}]: dispatch 2024-04-23T18:32:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:25 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e548975f-da5d-4394-a5c6-e658a6c89046"}]: dispatch 2024-04-23T18:32:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:25 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e548975f-da5d-4394-a5c6-e658a6c89046"}]': finished 2024-04-23T18:32:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:25 smithi148 ceph-mon[32311]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T18:32:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:25 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:25.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:25 smithi148 ceph-mon[32311]: from='client.? 172.21.15.148:0/710220498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:32:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:25 smithi157 ceph-mon[32580]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:25 smithi157 ceph-mon[32580]: from='client.? 172.21.15.148:0/4130567983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e548975f-da5d-4394-a5c6-e658a6c89046"}]: dispatch 2024-04-23T18:32:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:25 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e548975f-da5d-4394-a5c6-e658a6c89046"}]: dispatch 2024-04-23T18:32:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:25 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e548975f-da5d-4394-a5c6-e658a6c89046"}]': finished 2024-04-23T18:32:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:25 smithi157 ceph-mon[32580]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T18:32:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:25 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:25.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:25 smithi157 ceph-mon[32580]: from='client.? 172.21.15.148:0/710220498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:32:25.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:25 smithi132 ceph-mon[29178]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:25.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:25 smithi132 ceph-mon[29178]: from='client.? 172.21.15.148:0/4130567983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e548975f-da5d-4394-a5c6-e658a6c89046"}]: dispatch 2024-04-23T18:32:25.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:25 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e548975f-da5d-4394-a5c6-e658a6c89046"}]: dispatch 2024-04-23T18:32:25.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:25 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e548975f-da5d-4394-a5c6-e658a6c89046"}]': finished 2024-04-23T18:32:25.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:25 smithi132 ceph-mon[29178]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T18:32:25.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:25 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:25.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:25 smithi132 ceph-mon[29178]: from='client.? 172.21.15.148:0/710220498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:32:27.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:27 smithi148 ceph-mon[32311]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:27.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:27 smithi157 ceph-mon[32580]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:27.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:27 smithi132 ceph-mon[29178]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:29.420 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:29 smithi148 ceph-mon[32311]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:29.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:29 smithi157 ceph-mon[32580]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:29.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:29 smithi132 ceph-mon[29178]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:31.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:31 smithi148 ceph-mon[32311]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:31.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:31 smithi157 ceph-mon[32580]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:31.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:31 smithi132 ceph-mon[29178]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:32.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T18:32:32.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:32.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T18:32:32.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:32.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T18:32:32.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:33.441 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:33 smithi157 ceph-mon[32580]: Deploying daemon osd.10 on smithi148 2024-04-23T18:32:33.441 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:33 smithi157 ceph-mon[32580]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:33.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:33 smithi148 ceph-mon[32311]: Deploying daemon osd.10 on smithi148 2024-04-23T18:32:33.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:33 smithi148 ceph-mon[32311]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:33.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:33 smithi132 ceph-mon[29178]: Deploying daemon osd.10 on smithi148 2024-04-23T18:32:33.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:33 smithi132 ceph-mon[29178]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:35.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:35 smithi148 ceph-mon[32311]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:35.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:35 smithi157 ceph-mon[32580]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:35.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:35 smithi132 ceph-mon[29178]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:37.440 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:37 smithi148 ceph-mon[32311]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:37.542 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:37 smithi157 ceph-mon[32580]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:37.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:37 smithi132 ceph-mon[29178]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:39.092 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:39.092 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:39.092 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:39.093 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:38 smithi148 ceph-mon[32311]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:39.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:39.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:39.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:39.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:38 smithi132 ceph-mon[29178]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:39.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:39.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:39.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:39.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:38 smithi157 ceph-mon[32580]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:41.478 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:41 smithi148 ceph-mon[32311]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:41.478 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:41 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:41.478 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:41 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:41.478 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:41 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:41.478 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:41 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:41.478 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:41 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:41 smithi157 ceph-mon[32580]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:41 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:41 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:41 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:41 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:41 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:41.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:41 smithi132 ceph-mon[29178]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:41.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:41 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:41.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:41 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:41.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:41 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:41.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:41 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:41.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:41 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:41.864 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 10 on host 'smithi148' 2024-04-23T18:32:41.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.872+0000 7f617d7fa700 1 -- 172.21.15.148:0/217537558 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f618006ae30 con 0x7f616c0a3cc0 2024-04-23T18:32:41.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.875+0000 7f6185f08700 1 -- 172.21.15.148:0/217537558 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f616c0a3cc0 msgr2=0x7f616c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:41.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.875+0000 7f6185f08700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f616c0a3cc0 0x7f616c0a6180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6170000c00 tx=0x7f61700036c0 comp rx=0 tx=0).stop 2024-04-23T18:32:41.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.875+0000 7f6185f08700 1 -- 172.21.15.148:0/217537558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6180103540 msgr2=0x7f61801a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:41.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.875+0000 7f6185f08700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6180103540 0x7f61801a0220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6178007f10 tx=0x7f617800d3b0 comp rx=0 tx=0).stop 2024-04-23T18:32:41.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.875+0000 7f6185f08700 1 -- 172.21.15.148:0/217537558 shutdown_connections 2024-04-23T18:32:41.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.875+0000 7f6185f08700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6180103540 0x7f61801a0220 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:41.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.875+0000 7f6185f08700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f616c0a3cc0 0x7f616c0a6180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:41.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.875+0000 7f6185f08700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61801019a0 0x7f6180198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:41.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.875+0000 7f6185f08700 1 --2- 172.21.15.148:0/217537558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6180102ba0 0x7f61801991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:41.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.875+0000 7f6185f08700 1 -- 172.21.15.148:0/217537558 >> 172.21.15.148:0/217537558 conn(0x7f61800fcf40 msgr2=0x7f618006a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:41.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.875+0000 7f6185f08700 1 -- 172.21.15.148:0/217537558 shutdown_connections 2024-04-23T18:32:41.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T18:32:41.875+0000 7f6185f08700 1 -- 172.21.15.148:0/217537558 wait complete. 2024-04-23T18:32:42.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:42 smithi148 ceph-mon[32311]: from='osd.10 [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:32:42.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:42 smithi148 ceph-mon[32311]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:32:42.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:42 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:42.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:42 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:42.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:42 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:42 smithi157 ceph-mon[32580]: from='osd.10 [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:32:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:42 smithi157 ceph-mon[32580]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:32:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:42 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:42 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:42 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:42.577 DEBUG:teuthology.orchestra.run.smithi148:osd.10> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.10.service 2024-04-23T18:32:42.580 INFO:tasks.cephadm:Deploying osd.11 on smithi157 with /dev/vg_nvme/lv_1... 2024-04-23T18:32:42.580 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T18:32:42.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:42 smithi132 ceph-mon[29178]: from='osd.10 [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:32:42.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:42 smithi132 ceph-mon[29178]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:32:42.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:42 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:42.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:42 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:42.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:42 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:42.828 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:32:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:43 smithi157 ceph-mon[32580]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:43 smithi157 ceph-mon[32580]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T18:32:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:43 smithi157 ceph-mon[32580]: from='osd.10 [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:32:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:43 smithi157 ceph-mon[32580]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T18:32:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:43 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:43.276 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:43 smithi157 ceph-mon[32580]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:32:43.451 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:43 smithi148 ceph-mon[32311]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:43.451 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:43 smithi148 ceph-mon[32311]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T18:32:43.451 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:43 smithi148 ceph-mon[32311]: from='osd.10 [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:32:43.451 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:43 smithi148 ceph-mon[32311]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T18:32:43.451 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:43 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:43.451 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:43 smithi148 ceph-mon[32311]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:32:43.451 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:32:43 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:32:43.184+0000 7fae27fec700 -1 osd.10 0 waiting for initial osdmap 2024-04-23T18:32:43.451 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:32:43 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:32:43.195+0000 7fae23ddf700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:32:43.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:43 smithi132 ceph-mon[29178]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:43.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:43 smithi132 ceph-mon[29178]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T18:32:43.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:43 smithi132 ceph-mon[29178]: from='osd.10 [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:32:43.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:43 smithi132 ceph-mon[29178]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T18:32:43.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:43 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:43.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:43 smithi132 ceph-mon[29178]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-23T18:32:44.244 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:44 smithi157 ceph-mon[32580]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-23T18:32:44.244 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:44 smithi157 ceph-mon[32580]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T18:32:44.245 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:44.245 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:44.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:44 smithi148 ceph-mon[32311]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-23T18:32:44.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:44 smithi148 ceph-mon[32311]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T18:32:44.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:44 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:44.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:44 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:44.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:44 smithi132 ceph-mon[29178]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-23T18:32:44.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:44 smithi132 ceph-mon[29178]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T18:32:44.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:44.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:44.981 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T18:32:44.981 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T18:32:44.982 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-04-23T18:32:44.982 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-04-23T18:32:44.982 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00847622 s, 1.2 GB/s 2024-04-23T18:32:44.982 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-04-23T18:32:45.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:45 smithi157 ceph-mon[32580]: purged_snaps scrub starts 2024-04-23T18:32:45.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:45 smithi157 ceph-mon[32580]: purged_snaps scrub ok 2024-04-23T18:32:45.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:45 smithi157 ceph-mon[32580]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:45.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:45 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:45.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:45 smithi157 ceph-mon[32580]: osd.10 [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299] boot 2024-04-23T18:32:45.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:45 smithi157 ceph-mon[32580]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T18:32:45.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:45 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:45.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:45 smithi148 ceph-mon[32311]: purged_snaps scrub starts 2024-04-23T18:32:45.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:45 smithi148 ceph-mon[32311]: purged_snaps scrub ok 2024-04-23T18:32:45.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:45 smithi148 ceph-mon[32311]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:45.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:45 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:45.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:45 smithi148 ceph-mon[32311]: osd.10 [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299] boot 2024-04-23T18:32:45.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:45 smithi148 ceph-mon[32311]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T18:32:45.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:45 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:45.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:45 smithi132 ceph-mon[29178]: purged_snaps scrub starts 2024-04-23T18:32:45.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:45 smithi132 ceph-mon[29178]: purged_snaps scrub ok 2024-04-23T18:32:45.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:45 smithi132 ceph-mon[29178]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:32:45.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:45 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:45.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:45 smithi132 ceph-mon[29178]: osd.10 [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299] boot 2024-04-23T18:32:45.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:45 smithi132 ceph-mon[29178]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T18:32:45.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:45 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:32:45.668 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_1 2024-04-23T18:32:45.913 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:32:46.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:46 smithi157 ceph-mon[32580]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T18:32:46.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:46.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:46.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:46.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:46.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:46.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:46.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:46.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:46.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:46.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:46.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:46 smithi148 ceph-mon[32311]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T18:32:46.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:46.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:46.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:46 smithi132 ceph-mon[29178]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T18:32:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:32:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:46.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:32:47.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:47 smithi157 ceph-mon[32580]: Detected new or changed devices on smithi148 2024-04-23T18:32:47.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:47 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi148 to 4272M 2024-04-23T18:32:47.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:47 smithi157 ceph-mon[32580]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:32:47.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:47 smithi157 ceph-mon[32580]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T18:32:47.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:47 smithi148 ceph-mon[32311]: Detected new or changed devices on smithi148 2024-04-23T18:32:47.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:47 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi148 to 4272M 2024-04-23T18:32:47.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:47 smithi148 ceph-mon[32311]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:32:47.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:47 smithi148 ceph-mon[32311]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T18:32:47.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:47 smithi132 ceph-mon[29178]: Detected new or changed devices on smithi148 2024-04-23T18:32:47.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:47 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi148 to 4272M 2024-04-23T18:32:47.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:47 smithi132 ceph-mon[29178]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:32:47.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:47 smithi132 ceph-mon[29178]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T18:32:47.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.756+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2177183081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca0104640 msgr2=0x7f6ca010aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:47.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.756+0000 7f6ca4f96700 1 --2- 172.21.15.157:0/2177183081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca0104640 0x7f6ca010aef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800b0a0 tx=0x7f6c9800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:32:47.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.756+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2177183081 shutdown_connections 2024-04-23T18:32:47.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.756+0000 7f6ca4f96700 1 --2- 172.21.15.157:0/2177183081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca0104640 0x7f6ca010aef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:47.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.756+0000 7f6ca4f96700 1 --2- 172.21.15.157:0/2177183081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca0102aa0 0x7f6ca0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:47.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.756+0000 7f6ca4f96700 1 --2- 172.21.15.157:0/2177183081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca0103ca0 0x7f6ca0104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:47.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.756+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2177183081 >> 172.21.15.157:0/2177183081 conn(0x7f6ca00fe040 msgr2=0x7f6ca0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:47.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.756+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2177183081 shutdown_connections 2024-04-23T18:32:47.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.756+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2177183081 wait complete. 2024-04-23T18:32:47.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6ca4f96700 1 Processor -- start 2024-04-23T18:32:47.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6ca4f96700 1 -- start start 2024-04-23T18:32:47.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6ca4f96700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca0102aa0 0x7f6ca00767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:47.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6ca4f96700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca0103ca0 0x7f6ca0074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:47.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6ca4f96700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca0104640 0x7f6ca0075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:47.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6ca4f96700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca010e520 con 0x7f6ca0104640 2024-04-23T18:32:47.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6ca4f96700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca010e3a0 con 0x7f6ca0102aa0 2024-04-23T18:32:47.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6ca4f96700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca010e6a0 con 0x7f6ca0103ca0 2024-04-23T18:32:47.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6c9dd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca0103ca0 0x7f6ca0074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6c9dd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca0103ca0 0x7f6ca0074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34960/0 (socket says 172.21.15.157:34960) 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6c9dd9b700 1 -- 172.21.15.157:0/2426150068 learned_addr learned my addr 172.21.15.157:0/2426150068 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6c9e59c700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca0102aa0 0x7f6ca00767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6c9ed9d700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca0104640 0x7f6ca0075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6c9dd9b700 1 -- 172.21.15.157:0/2426150068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca0102aa0 msgr2=0x7f6ca00767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6c9dd9b700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca0102aa0 0x7f6ca00767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6c9dd9b700 1 -- 172.21.15.157:0/2426150068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca0104640 msgr2=0x7f6ca0075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6c9dd9b700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca0104640 0x7f6ca0075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.757+0000 7f6c9dd9b700 1 -- 172.21.15.157:0/2426150068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca0075c20 con 0x7f6ca0103ca0 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.758+0000 7f6c9dd9b700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca0103ca0 0x7f6ca0074e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400d950 tx=0x7f6c9400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.758+0000 7f6c8f7fe700 1 -- 172.21.15.157:0/2426150068 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94016070 con 0x7f6ca0103ca0 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.758+0000 7f6c8f7fe700 1 -- 172.21.15.157:0/2426150068 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9400fd40 con 0x7f6ca0103ca0 2024-04-23T18:32:47.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.758+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2426150068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca01a2cf0 con 0x7f6ca0103ca0 2024-04-23T18:32:47.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.758+0000 7f6c8f7fe700 1 -- 172.21.15.157:0/2426150068 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400e3b0 con 0x7f6ca0103ca0 2024-04-23T18:32:47.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.758+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2426150068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca01a31b0 con 0x7f6ca0103ca0 2024-04-23T18:32:47.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.759+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2426150068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca0069730 con 0x7f6ca0103ca0 2024-04-23T18:32:47.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.760+0000 7f6c8f7fe700 1 -- 172.21.15.157:0/2426150068 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f6c94019070 con 0x7f6ca0103ca0 2024-04-23T18:32:47.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.760+0000 7f6c8f7fe700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6c880a3c40 0x7f6c880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:47.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.761+0000 7f6c8f7fe700 1 -- 172.21.15.157:0/2426150068 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f6c940cf820 con 0x7f6ca0103ca0 2024-04-23T18:32:47.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.761+0000 7f6c9e59c700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6c880a3c40 0x7f6c880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:47.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.761+0000 7f6c9e59c700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6c880a3c40 0x7f6c880a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0103b00 tx=0x7f6c9000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:47.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.764+0000 7f6c8f7fe700 1 -- 172.21.15.157:0/2426150068 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c9409c850 con 0x7f6ca0103ca0 2024-04-23T18:32:47.966 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:32:47.974+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2426150068 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6ca0062e80 con 0x7f6c880a3c40 2024-04-23T18:32:48.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:48 smithi148 ceph-mon[32311]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T18:32:48.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:48 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:32:48.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:48 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:32:48.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:48 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:48.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:48 smithi157 ceph-mon[32580]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T18:32:48.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:48 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:32:48.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:48 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:32:48.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:48 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:48.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:48 smithi132 ceph-mon[29178]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T18:32:48.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:48 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:32:48.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:48 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:32:48.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:48 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:49.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:49 smithi148 ceph-mon[32311]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:32:49.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:49 smithi148 ceph-mon[32311]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:32:49.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:49 smithi148 ceph-mon[32311]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T18:32:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:49 smithi157 ceph-mon[32580]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:32:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:49 smithi157 ceph-mon[32580]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:32:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:49 smithi157 ceph-mon[32580]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T18:32:49.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:49 smithi132 ceph-mon[29178]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:32:49.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:49 smithi132 ceph-mon[29178]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:32:49.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:49 smithi132 ceph-mon[29178]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T18:32:51.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:51 smithi148 ceph-mon[32311]: pgmap v245: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-23T18:32:51.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:51 smithi148 ceph-mon[32311]: from='client.? 172.21.15.157:0/2983891457' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d9c5bae-d624-411c-8636-8156934d51fe"}]: dispatch 2024-04-23T18:32:51.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:51 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d9c5bae-d624-411c-8636-8156934d51fe"}]: dispatch 2024-04-23T18:32:51.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:51 smithi148 ceph-mon[32311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d9c5bae-d624-411c-8636-8156934d51fe"}]': finished 2024-04-23T18:32:51.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:51 smithi148 ceph-mon[32311]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T18:32:51.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:51 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:32:51.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:51 smithi157 ceph-mon[32580]: pgmap v245: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-23T18:32:51.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:51 smithi157 ceph-mon[32580]: from='client.? 172.21.15.157:0/2983891457' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d9c5bae-d624-411c-8636-8156934d51fe"}]: dispatch 2024-04-23T18:32:51.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:51 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d9c5bae-d624-411c-8636-8156934d51fe"}]: dispatch 2024-04-23T18:32:51.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:51 smithi157 ceph-mon[32580]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d9c5bae-d624-411c-8636-8156934d51fe"}]': finished 2024-04-23T18:32:51.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:51 smithi157 ceph-mon[32580]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T18:32:51.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:51 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:32:51.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:51 smithi132 ceph-mon[29178]: pgmap v245: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-23T18:32:51.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:51 smithi132 ceph-mon[29178]: from='client.? 172.21.15.157:0/2983891457' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d9c5bae-d624-411c-8636-8156934d51fe"}]: dispatch 2024-04-23T18:32:51.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:51 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d9c5bae-d624-411c-8636-8156934d51fe"}]: dispatch 2024-04-23T18:32:51.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:51 smithi132 ceph-mon[29178]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d9c5bae-d624-411c-8636-8156934d51fe"}]': finished 2024-04-23T18:32:51.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:51 smithi132 ceph-mon[29178]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T18:32:51.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:51 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:32:52.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:52 smithi148 ceph-mon[32311]: from='client.? 172.21.15.157:0/3258767036' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:32:52.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:52 smithi157 ceph-mon[32580]: from='client.? 172.21.15.157:0/3258767036' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:32:52.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:52 smithi132 ceph-mon[29178]: from='client.? 172.21.15.157:0/3258767036' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:32:53.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:53 smithi148 ceph-mon[32311]: pgmap v247: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-23T18:32:53.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:53 smithi157 ceph-mon[32580]: pgmap v247: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-23T18:32:53.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:53 smithi132 ceph-mon[29178]: pgmap v247: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-23T18:32:54.361 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:32:54 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:32:54.066+0000 7f6be7e3e700 -1 calc_pg_upmaps abort due to max <= 0 2024-04-23T18:32:55.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:55 smithi148 ceph-mon[32311]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:32:55.542 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:55 smithi157 ceph-mon[32580]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:32:55.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:55 smithi132 ceph-mon[29178]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:32:57.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:57 smithi157 ceph-mon[32580]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:32:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:57 smithi148 ceph-mon[32311]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:32:57.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:57 smithi132 ceph-mon[29178]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:32:59.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:59 smithi157 ceph-mon[32580]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:32:59.243 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T18:32:59.243 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:32:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:59.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:59 smithi148 ceph-mon[32311]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:32:59.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T18:32:59.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:32:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:59.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:59 smithi132 ceph-mon[29178]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:32:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T18:32:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:32:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:00.286 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:00 smithi157 ceph-mon[32580]: Deploying daemon osd.11 on smithi157 2024-04-23T18:33:00.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:00 smithi148 ceph-mon[32311]: Deploying daemon osd.11 on smithi157 2024-04-23T18:33:00.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:00 smithi132 ceph-mon[29178]: Deploying daemon osd.11 on smithi157 2024-04-23T18:33:01.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:01 smithi148 ceph-mon[32311]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:01.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:01 smithi157 ceph-mon[32580]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:01.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:01 smithi132 ceph-mon[29178]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:03.486 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:03 smithi157 ceph-mon[32580]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:03.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:03 smithi148 ceph-mon[32311]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:03.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:03 smithi132 ceph-mon[29178]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:05.464 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:05 smithi157 ceph-mon[32580]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:05.464 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:05 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:05.464 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:05 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:05.464 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:05 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:05.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:05 smithi148 ceph-mon[32311]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:05.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:05 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:05.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:05 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:05.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:05 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:05.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:05 smithi132 ceph-mon[29178]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:05.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:05 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:05.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:05 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:05.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:05 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:07.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:07 smithi157 ceph-mon[32580]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:07.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:07 smithi148 ceph-mon[32311]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:07.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:07 smithi132 ceph-mon[29178]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:07.831 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.838+0000 7f6c8f7fe700 1 -- 172.21.15.157:0/2426150068 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6ca0062e80 con 0x7f6c880a3c40 2024-04-23T18:33:07.831 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 11 on host 'smithi157' 2024-04-23T18:33:07.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.840+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2426150068 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6c880a3c40 msgr2=0x7f6c880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:07.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.840+0000 7f6ca4f96700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6c880a3c40 0x7f6c880a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0103b00 tx=0x7f6c9000b410 comp rx=0 tx=0).stop 2024-04-23T18:33:07.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.840+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2426150068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca0103ca0 msgr2=0x7f6ca0074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:07.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.840+0000 7f6ca4f96700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca0103ca0 0x7f6ca0074e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400d950 tx=0x7f6c9400dc60 comp rx=0 tx=0).stop 2024-04-23T18:33:07.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.840+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2426150068 shutdown_connections 2024-04-23T18:33:07.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.840+0000 7f6ca4f96700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6c880a3c40 0x7f6c880a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:07.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.840+0000 7f6ca4f96700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca0104640 0x7f6ca0075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:07.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.840+0000 7f6ca4f96700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca0102aa0 0x7f6ca00767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:07.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.840+0000 7f6ca4f96700 1 --2- 172.21.15.157:0/2426150068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca0103ca0 0x7f6ca0074e30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:07.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.840+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2426150068 >> 172.21.15.157:0/2426150068 conn(0x7f6ca00fe040 msgr2=0x7f6ca0106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:07.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.840+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2426150068 shutdown_connections 2024-04-23T18:33:07.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:07.840+0000 7f6ca4f96700 1 -- 172.21.15.157:0/2426150068 wait complete. 2024-04-23T18:33:08.846 DEBUG:teuthology.orchestra.run.smithi157:osd.11> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.11.service 2024-04-23T18:33:08.848 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-23T18:33:08.849 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-23T18:33:09.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:08 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:09.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:08 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:09.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:08 smithi148 ceph-mon[32311]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:09.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:08 smithi148 ceph-mon[32311]: from='osd.11 [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:33:09.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:08 smithi148 ceph-mon[32311]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:33:09.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:08 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:09.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:08 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:09.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:08 smithi157 ceph-mon[32580]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:09.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:08 smithi157 ceph-mon[32580]: from='osd.11 [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:33:09.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:08 smithi157 ceph-mon[32580]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:33:09.097 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:33:09.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:08 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:09.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:08 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:09.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:08 smithi132 ceph-mon[29178]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:09.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:08 smithi132 ceph-mon[29178]: from='osd.11 [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:33:09.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:08 smithi132 ceph-mon[29178]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:33:09.900 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:09 smithi157 ceph-mon[32580]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T18:33:09.900 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:09 smithi157 ceph-mon[32580]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T18:33:09.900 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:09 smithi157 ceph-mon[32580]: from='osd.11 [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:33:09.900 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:09.900 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:09 smithi157 ceph-mon[32580]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:33:09.900 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:09.900 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:09 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:09.900 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:33:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:33:09.813+0000 7f0aae566700 -1 osd.11 0 waiting for initial osdmap 2024-04-23T18:33:09.900 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:33:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:33:09.828+0000 7f0aaa359700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:33:10.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:09 smithi132 ceph-mon[29178]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T18:33:10.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:09 smithi132 ceph-mon[29178]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T18:33:10.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:09 smithi132 ceph-mon[29178]: from='osd.11 [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:33:10.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:10.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:09 smithi132 ceph-mon[29178]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:33:10.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:10.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:09 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:10.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:09 smithi148 ceph-mon[32311]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T18:33:10.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:09 smithi148 ceph-mon[32311]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T18:33:10.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:09 smithi148 ceph-mon[32311]: from='osd.11 [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:33:10.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:10.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:09 smithi148 ceph-mon[32311]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-23T18:33:10.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:10.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:09 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:10.850 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:10 smithi132 ceph-mon[29178]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-23T18:33:10.850 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:10 smithi132 ceph-mon[29178]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T18:33:10.851 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:10.851 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:10 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:10.851 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:10 smithi132 ceph-mon[29178]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:10.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.958+0000 7f1bfae39700 1 -- 172.21.15.132:0/2778482607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf4104640 msgr2=0x7f1bf410aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:10.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.958+0000 7f1bfae39700 1 --2- 172.21.15.132:0/2778482607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf4104640 0x7f1bf410aef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00b0d0 tx=0x7f1bec00b3e0 comp rx=0 tx=0).stop 2024-04-23T18:33:10.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.958+0000 7f1bfae39700 1 -- 172.21.15.132:0/2778482607 shutdown_connections 2024-04-23T18:33:10.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.958+0000 7f1bfae39700 1 --2- 172.21.15.132:0/2778482607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf4104640 0x7f1bf410aef0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:10.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.958+0000 7f1bfae39700 1 --2- 172.21.15.132:0/2778482607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf4102aa0 0x7f1bf4102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:10.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.958+0000 7f1bfae39700 1 --2- 172.21.15.132:0/2778482607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf4103ca0 0x7f1bf4104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:10.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.958+0000 7f1bfae39700 1 -- 172.21.15.132:0/2778482607 >> 172.21.15.132:0/2778482607 conn(0x7f1bf40fe040 msgr2=0x7f1bf4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:10.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.958+0000 7f1bfae39700 1 -- 172.21.15.132:0/2778482607 shutdown_connections 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.958+0000 7f1bfae39700 1 -- 172.21.15.132:0/2778482607 wait complete. 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.958+0000 7f1bfae39700 1 Processor -- start 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.958+0000 7f1bfae39700 1 -- start start 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bfae39700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf4102aa0 0x7f1bf4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bfae39700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf4103ca0 0x7f1bf41990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bfae39700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf4104640 0x7f1bf41a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bfae39700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf410e520 con 0x7f1bf4104640 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bfae39700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf410e3a0 con 0x7f1bf4103ca0 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bfae39700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf410e6a0 con 0x7f1bf4102aa0 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf93d6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf4104640 0x7f1bf41a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf93d6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf4104640 0x7f1bf41a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48902/0 (socket says 172.21.15.132:48902) 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf93d6700 1 -- 172.21.15.132:0/351913183 learned_addr learned my addr 172.21.15.132:0/351913183 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf3fff700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf4103ca0 0x7f1bf41990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf8bd5700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf4102aa0 0x7f1bf4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf93d6700 1 -- 172.21.15.132:0/351913183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf4102aa0 msgr2=0x7f1bf4198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf93d6700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf4102aa0 0x7f1bf4198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf93d6700 1 -- 172.21.15.132:0/351913183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf4103ca0 msgr2=0x7f1bf41990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf93d6700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf4103ca0 0x7f1bf41990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf93d6700 1 -- 172.21.15.132:0/351913183 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf41a07c0 con 0x7f1bf4104640 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf8bd5700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf4102aa0 0x7f1bf4198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf93d6700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf4104640 0x7f1bf41a0120 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009fd0 tx=0x7f1bec004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf1ffb700 1 -- 172.21.15.132:0/351913183 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec01f030 con 0x7f1bf4104640 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf1ffb700 1 -- 172.21.15.132:0/351913183 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bec003a00 con 0x7f1bf4104640 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bfae39700 1 -- 172.21.15.132:0/351913183 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf41a0a50 con 0x7f1bf4104640 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.959+0000 7f1bf1ffb700 1 -- 172.21.15.132:0/351913183 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec0046a0 con 0x7f1bf4104640 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.960+0000 7f1bfae39700 1 -- 172.21.15.132:0/351913183 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf41a0ea0 con 0x7f1bf4104640 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.961+0000 7f1bfae39700 1 -- 172.21.15.132:0/351913183 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf402df80 con 0x7f1bf4104640 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.962+0000 7f1bf1ffb700 1 -- 172.21.15.132:0/351913183 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1bec003bb0 con 0x7f1bf4104640 2024-04-23T18:33:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.963+0000 7f1bf1ffb700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1be40a3c40 0x7f1be40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:10.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.963+0000 7f1bf8bd5700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1be40a3c40 0x7f1be40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:10.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.963+0000 7f1bf1ffb700 1 -- 172.21.15.132:0/351913183 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1bec00f020 con 0x7f1bf4104640 2024-04-23T18:33:10.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.963+0000 7f1bf8bd5700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1be40a3c40 0x7f1be40a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4103b00 tx=0x7f1be800b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:10.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:10.967+0000 7f1bf1ffb700 1 -- 172.21.15.132:0/351913183 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bec029020 con 0x7f1bf4104640 2024-04-23T18:33:10.992 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:10 smithi157 ceph-mon[32580]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-23T18:33:10.992 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:10 smithi157 ceph-mon[32580]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T18:33:10.992 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:10.992 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:10 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:10.992 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:10 smithi157 ceph-mon[32580]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:11.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.163+0000 7f1bfae39700 1 -- 172.21.15.132:0/351913183 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1bf40636f0 con 0x7f1bf4104640 2024-04-23T18:33:11.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.163+0000 7f1bf1ffb700 1 -- 172.21.15.132:0/351913183 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f1bec09e500 con 0x7f1bf4104640 2024-04-23T18:33:11.157 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:11.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.165+0000 7f1bfae39700 1 -- 172.21.15.132:0/351913183 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1be40a3c40 msgr2=0x7f1be40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:11.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.165+0000 7f1bfae39700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1be40a3c40 0x7f1be40a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4103b00 tx=0x7f1be800b670 comp rx=0 tx=0).stop 2024-04-23T18:33:11.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.165+0000 7f1bfae39700 1 -- 172.21.15.132:0/351913183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf4104640 msgr2=0x7f1bf41a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:11.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.165+0000 7f1bfae39700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf4104640 0x7f1bf41a0120 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009fd0 tx=0x7f1bec004d50 comp rx=0 tx=0).stop 2024-04-23T18:33:11.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.165+0000 7f1bfae39700 1 -- 172.21.15.132:0/351913183 shutdown_connections 2024-04-23T18:33:11.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.165+0000 7f1bfae39700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1be40a3c40 0x7f1be40a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.165+0000 7f1bfae39700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf4104640 0x7f1bf41a0120 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.165+0000 7f1bfae39700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf4102aa0 0x7f1bf4198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.165+0000 7f1bfae39700 1 --2- 172.21.15.132:0/351913183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf4103ca0 0x7f1bf41990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:11.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.165+0000 7f1bfae39700 1 -- 172.21.15.132:0/351913183 >> 172.21.15.132:0/351913183 conn(0x7f1bf40fe040 msgr2=0x7f1bf4107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:11.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.166+0000 7f1bfae39700 1 -- 172.21.15.132:0/351913183 shutdown_connections 2024-04-23T18:33:11.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:11.166+0000 7f1bfae39700 1 -- 172.21.15.132:0/351913183 wait complete. 2024-04-23T18:33:11.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:10 smithi148 ceph-mon[32311]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-23T18:33:11.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:10 smithi148 ceph-mon[32311]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T18:33:11.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:11.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:10 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:11.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:10 smithi148 ceph-mon[32311]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:33:11.865 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713897190,"num_in_osds":12,"osd_in_since":1713897170,"num_remapped_pgs":0} 2024-04-23T18:33:11.866 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T18:33:12.108 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:33:12.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:11 smithi132 ceph-mon[29178]: purged_snaps scrub starts 2024-04-23T18:33:12.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:11 smithi132 ceph-mon[29178]: purged_snaps scrub ok 2024-04-23T18:33:12.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:11 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:12.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:11 smithi132 ceph-mon[29178]: osd.11 [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818] boot 2024-04-23T18:33:12.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:11 smithi132 ceph-mon[29178]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T18:33:12.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:11 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:12.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:11 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/351913183' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:33:12.171 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:11 smithi157 ceph-mon[32580]: purged_snaps scrub starts 2024-04-23T18:33:12.171 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:11 smithi157 ceph-mon[32580]: purged_snaps scrub ok 2024-04-23T18:33:12.171 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:11 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:12.171 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:11 smithi157 ceph-mon[32580]: osd.11 [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818] boot 2024-04-23T18:33:12.171 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:11 smithi157 ceph-mon[32580]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T18:33:12.171 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:11 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:12.171 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:11 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/351913183' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:33:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:11 smithi148 ceph-mon[32311]: purged_snaps scrub starts 2024-04-23T18:33:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:11 smithi148 ceph-mon[32311]: purged_snaps scrub ok 2024-04-23T18:33:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:11 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:11 smithi148 ceph-mon[32311]: osd.11 [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818] boot 2024-04-23T18:33:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:11 smithi148 ceph-mon[32311]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T18:33:12.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:11 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:33:12.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:11 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/351913183' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:13.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:12 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:13.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T18:33:13.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:13.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:13.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:13.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:33:13.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:33:13.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:33:13.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:33:13.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:13.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:13.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:13.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:13.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:12 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:13.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T18:33:13.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:13.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:13.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:13.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:33:13.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:33:13.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:33:13.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:33:13.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:13.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:13.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:13.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:13.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:12 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:13.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.786+0000 7f25b1f23700 1 -- 172.21.15.132:0/1259307109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25ac101ff0 msgr2=0x7f25ac105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:13.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.786+0000 7f25b1f23700 1 --2- 172.21.15.132:0/1259307109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25ac101ff0 0x7f25ac105db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009990 tx=0x7f25a0009ca0 comp rx=0 tx=0).stop 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.787+0000 7f25b1f23700 1 -- 172.21.15.132:0/1259307109 shutdown_connections 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.787+0000 7f25b1f23700 1 --2- 172.21.15.132:0/1259307109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ac1064e0 0x7f25ac10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.787+0000 7f25b1f23700 1 --2- 172.21.15.132:0/1259307109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac101640 0x7f25ac101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.787+0000 7f25b1f23700 1 --2- 172.21.15.132:0/1259307109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25ac101ff0 0x7f25ac105db0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.787+0000 7f25b1f23700 1 -- 172.21.15.132:0/1259307109 >> 172.21.15.132:0/1259307109 conn(0x7f25ac076c60 msgr2=0x7f25ac077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.787+0000 7f25b1f23700 1 -- 172.21.15.132:0/1259307109 shutdown_connections 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.787+0000 7f25b1f23700 1 -- 172.21.15.132:0/1259307109 wait complete. 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.787+0000 7f25b1f23700 1 Processor -- start 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.787+0000 7f25b1f23700 1 -- start start 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25b1f23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25ac101640 0x7f25ac19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25b1f23700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac101ff0 0x7f25ac19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25b1f23700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ac1064e0 0x7f25ac1a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25b1f23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac111ba0 con 0x7f25ac101640 2024-04-23T18:33:13.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25b1f23700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac111a20 con 0x7f25ac101ff0 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25b1f23700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac111d20 con 0x7f25ac1064e0 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25ab7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25ac101640 0x7f25ac19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25ab7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25ac101640 0x7f25ac19a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48922/0 (socket says 172.21.15.132:48922) 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25ab7fe700 1 -- 172.21.15.132:0/2325371543 learned_addr learned my addr 172.21.15.132:0/2325371543 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25abfff700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ac1064e0 0x7f25ac1a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25aaffd700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac101ff0 0x7f25ac19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25ab7fe700 1 -- 172.21.15.132:0/2325371543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ac1064e0 msgr2=0x7f25ac1a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25ab7fe700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ac1064e0 0x7f25ac1a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25ab7fe700 1 -- 172.21.15.132:0/2325371543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac101ff0 msgr2=0x7f25ac19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25ab7fe700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac101ff0 0x7f25ac19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25ab7fe700 1 -- 172.21.15.132:0/2325371543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac1a55c0 con 0x7f25ac101640 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25aaffd700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac101ff0 0x7f25ac19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25ab7fe700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25ac101640 0x7f25ac19a000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f259c00d980 tx=0x7f259c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25a8ff9700 1 -- 172.21.15.132:0/2325371543 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c016070 con 0x7f25ac101640 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25b1f23700 1 -- 172.21.15.132:0/2325371543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ac1a57f0 con 0x7f25ac101640 2024-04-23T18:33:13.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25a8ff9700 1 -- 172.21.15.132:0/2325371543 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f259c0044d0 con 0x7f25ac101640 2024-04-23T18:33:13.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25a8ff9700 1 -- 172.21.15.132:0/2325371543 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c004d00 con 0x7f25ac101640 2024-04-23T18:33:13.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.788+0000 7f25b1f23700 1 -- 172.21.15.132:0/2325371543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ac1a5d00 con 0x7f25ac101640 2024-04-23T18:33:13.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.789+0000 7f25b1f23700 1 -- 172.21.15.132:0/2325371543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ac069730 con 0x7f25ac101640 2024-04-23T18:33:13.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.790+0000 7f25a8ff9700 1 -- 172.21.15.132:0/2325371543 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f259c004020 con 0x7f25ac101640 2024-04-23T18:33:13.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.791+0000 7f25a8ff9700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f25940a3c40 0x7f25940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:13.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.791+0000 7f25aaffd700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f25940a3c40 0x7f25940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:13.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.791+0000 7f25a8ff9700 1 -- 172.21.15.132:0/2325371543 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f259c0cfe40 con 0x7f25ac101640 2024-04-23T18:33:13.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.792+0000 7f25aaffd700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f25940a3c40 0x7f25940a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f25ac10f460 tx=0x7f25a001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:13.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.796+0000 7f25a8ff9700 1 -- 172.21.15.132:0/2325371543 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f259c01f020 con 0x7f25ac101640 2024-04-23T18:33:13.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.996+0000 7f25b1f23700 1 -- 172.21.15.132:0/2325371543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25ac02df80 con 0x7f25ac101640 2024-04-23T18:33:13.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.997+0000 7f25a8ff9700 1 -- 172.21.15.132:0/2325371543 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18538 (secure 0 0 0) 0x7f259c09ccc0 con 0x7f25ac101640 2024-04-23T18:33:13.990 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:13.991 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":82,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:33:11.819374+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:33:13.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.999+0000 7f25b1f23700 1 -- 172.21.15.132:0/2325371543 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f25940a3c40 msgr2=0x7f25940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:13.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.999+0000 7f25b1f23700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f25940a3c40 0x7f25940a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f25ac10f460 tx=0x7f25a001a040 comp rx=0 tx=0).stop 2024-04-23T18:33:13.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.999+0000 7f25b1f23700 1 -- 172.21.15.132:0/2325371543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25ac101640 msgr2=0x7f25ac19a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:13.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.999+0000 7f25b1f23700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25ac101640 0x7f25ac19a000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f259c00d980 tx=0x7f259c00dd40 comp rx=0 tx=0).stop 2024-04-23T18:33:13.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.999+0000 7f25b1f23700 1 -- 172.21.15.132:0/2325371543 shutdown_connections 2024-04-23T18:33:13.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.999+0000 7f25b1f23700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25ac1064e0 0x7f25ac1a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:13.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.999+0000 7f25b1f23700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f25940a3c40 0x7f25940a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:13.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.999+0000 7f25b1f23700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25ac101640 0x7f25ac19a000 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:13.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.999+0000 7f25b1f23700 1 --2- 172.21.15.132:0/2325371543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac101ff0 0x7f25ac19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:13.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.999+0000 7f25b1f23700 1 -- 172.21.15.132:0/2325371543 >> 172.21.15.132:0/2325371543 conn(0x7f25ac076c60 msgr2=0x7f25ac105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:13.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.999+0000 7f25b1f23700 1 -- 172.21.15.132:0/2325371543 shutdown_connections 2024-04-23T18:33:13.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:13.999+0000 7f25b1f23700 1 -- 172.21.15.132:0/2325371543 wait complete. 2024-04-23T18:33:14.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:13 smithi132 ceph-mon[29178]: Detected new or changed devices on smithi157 2024-04-23T18:33:14.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:13 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi157 to 4272M 2024-04-23T18:33:14.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:13 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:14.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:13 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:14.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:13 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:14.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:13 smithi148 ceph-mon[32311]: Detected new or changed devices on smithi157 2024-04-23T18:33:14.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:13 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi157 to 4272M 2024-04-23T18:33:14.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:13 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:14.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:13 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:14.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:13 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:14.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:13 smithi157 ceph-mon[32580]: Detected new or changed devices on smithi157 2024-04-23T18:33:14.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:13 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi157 to 4272M 2024-04-23T18:33:14.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:13 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:14.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:13 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:14.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:13 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:14.679 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:29:20.132451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:33:14.679 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-23T18:33:14.919 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:33:15.041 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:14 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2325371543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:33:15.041 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:14 smithi132 ceph-mon[29178]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:15.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:14 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2325371543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:33:15.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:14 smithi148 ceph-mon[32311]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:15.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:14 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2325371543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:33:15.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:14 smithi157 ceph-mon[32580]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:16.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.705+0000 7fad68a30700 1 -- 172.21.15.132:0/4251896143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad64103450 msgr2=0x7fad6410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.705+0000 7fad68a30700 1 --2- 172.21.15.132:0/4251896143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad64103450 0x7fad6410f8b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00b0a0 tx=0x7fad5c00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.705+0000 7fad68a30700 1 -- 172.21.15.132:0/4251896143 shutdown_connections 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.705+0000 7fad68a30700 1 --2- 172.21.15.132:0/4251896143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad64103450 0x7fad6410f8b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.705+0000 7fad68a30700 1 --2- 172.21.15.132:0/4251896143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad64108a50 0x7fad64108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.705+0000 7fad68a30700 1 --2- 172.21.15.132:0/4251896143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad64102a90 0x7fad64102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.705+0000 7fad68a30700 1 -- 172.21.15.132:0/4251896143 >> 172.21.15.132:0/4251896143 conn(0x7fad64079ba0 msgr2=0x7fad64079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.705+0000 7fad68a30700 1 -- 172.21.15.132:0/4251896143 shutdown_connections 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.705+0000 7fad68a30700 1 -- 172.21.15.132:0/4251896143 wait complete. 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.706+0000 7fad68a30700 1 Processor -- start 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.706+0000 7fad68a30700 1 -- start start 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.706+0000 7fad68a30700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad64102a90 0x7fad64199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.706+0000 7fad68a30700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad64103450 0x7fad641996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:16.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.706+0000 7fad68a30700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad64108a50 0x7fad6419da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.706+0000 7fad68a30700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fad64112c80 con 0x7fad64103450 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.706+0000 7fad68a30700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fad64112b00 con 0x7fad64102a90 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.706+0000 7fad68a30700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad64112e00 con 0x7fad64108a50 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.706+0000 7fad61d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad64103450 0x7fad641996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.706+0000 7fad61d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad64103450 0x7fad641996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48930/0 (socket says 172.21.15.132:48930) 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad61d9b700 1 -- 172.21.15.132:0/2252523061 learned_addr learned my addr 172.21.15.132:0/2252523061 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad62d9d700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad64108a50 0x7fad6419da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad6259c700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad64102a90 0x7fad64199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad61d9b700 1 -- 172.21.15.132:0/2252523061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad64108a50 msgr2=0x7fad6419da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad61d9b700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad64108a50 0x7fad6419da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad61d9b700 1 -- 172.21.15.132:0/2252523061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad64102a90 msgr2=0x7fad64199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad61d9b700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad64102a90 0x7fad64199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad61d9b700 1 -- 172.21.15.132:0/2252523061 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6419e150 con 0x7fad64103450 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad6259c700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad64102a90 0x7fad64199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:16.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad61d9b700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad64103450 0x7fad641996d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00b530 tx=0x7fad4c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:16.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad5b7fe700 1 -- 172.21.15.132:0/2252523061 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c015020 con 0x7fad64103450 2024-04-23T18:33:16.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad68a30700 1 -- 172.21.15.132:0/2252523061 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6419e440 con 0x7fad64103450 2024-04-23T18:33:16.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad5b7fe700 1 -- 172.21.15.132:0/2252523061 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad4c011ce0 con 0x7fad64103450 2024-04-23T18:33:16.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad5b7fe700 1 -- 172.21.15.132:0/2252523061 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c010d20 con 0x7fad64103450 2024-04-23T18:33:16.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.707+0000 7fad68a30700 1 -- 172.21.15.132:0/2252523061 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad641a5cc0 con 0x7fad64103450 2024-04-23T18:33:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.708+0000 7fad68a30700 1 -- 172.21.15.132:0/2252523061 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad64069730 con 0x7fad64103450 2024-04-23T18:33:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.709+0000 7fad5b7fe700 1 -- 172.21.15.132:0/2252523061 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fad4c018020 con 0x7fad64103450 2024-04-23T18:33:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.710+0000 7fad5b7fe700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fad500a3c40 0x7fad500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.710+0000 7fad6259c700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fad500a3c40 0x7fad500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.710+0000 7fad5b7fe700 1 -- 172.21.15.132:0/2252523061 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fad4c0cf7b0 con 0x7fad64103450 2024-04-23T18:33:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.711+0000 7fad6259c700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fad500a3c40 0x7fad500a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fad54005fd0 tx=0x7fad54005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:16.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.715+0000 7fad5b7fe700 1 -- 172.21.15.132:0/2252523061 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad4c01f020 con 0x7fad64103450 2024-04-23T18:33:16.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.910+0000 7fad68a30700 1 -- 172.21.15.132:0/2252523061 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fad6419a4a0 con 0x7fad64103450 2024-04-23T18:33:16.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.910+0000 7fad5b7fe700 1 -- 172.21.15.132:0/2252523061 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fad4c09c630 con 0x7fad64103450 2024-04-23T18:33:16.902 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-04-23T18:33:16.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.912+0000 7fad68a30700 1 -- 172.21.15.132:0/2252523061 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fad500a3c40 msgr2=0x7fad500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:16.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.912+0000 7fad68a30700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fad500a3c40 0x7fad500a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fad54005fd0 tx=0x7fad54005ee0 comp rx=0 tx=0).stop 2024-04-23T18:33:16.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.912+0000 7fad68a30700 1 -- 172.21.15.132:0/2252523061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad64103450 msgr2=0x7fad641996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:16.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.912+0000 7fad68a30700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad64103450 0x7fad641996d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00b530 tx=0x7fad4c00b8f0 comp rx=0 tx=0).stop 2024-04-23T18:33:16.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.912+0000 7fad68a30700 1 -- 172.21.15.132:0/2252523061 shutdown_connections 2024-04-23T18:33:16.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.912+0000 7fad68a30700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad64108a50 0x7fad6419da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:16.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.912+0000 7fad68a30700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad64102a90 0x7fad64199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:16.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.912+0000 7fad68a30700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fad500a3c40 0x7fad500a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:16.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.912+0000 7fad68a30700 1 --2- 172.21.15.132:0/2252523061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad64103450 0x7fad641996d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:16.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.912+0000 7fad68a30700 1 -- 172.21.15.132:0/2252523061 >> 172.21.15.132:0/2252523061 conn(0x7fad64079ba0 msgr2=0x7fad64101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:16.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.912+0000 7fad68a30700 1 -- 172.21.15.132:0/2252523061 shutdown_connections 2024-04-23T18:33:16.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:16.912+0000 7fad68a30700 1 -- 172.21.15.132:0/2252523061 wait complete. 2024-04-23T18:33:17.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:17 smithi148 ceph-mon[32311]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:17.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:17 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2252523061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:33:17.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:17 smithi157 ceph-mon[32580]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:17.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:17 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2252523061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:33:17.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:17 smithi132 ceph-mon[29178]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:17.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:17 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2252523061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:33:17.733 INFO:tasks.cephadm:Adding mds.a on smithi132 2024-04-23T18:33:17.733 INFO:tasks.cephadm:Adding mds.d on smithi132 2024-04-23T18:33:17.733 INFO:tasks.cephadm:Adding mds.g on smithi132 2024-04-23T18:33:17.733 INFO:tasks.cephadm:Adding mds.j on smithi132 2024-04-23T18:33:17.733 INFO:tasks.cephadm:Adding mds.b on smithi148 2024-04-23T18:33:17.733 INFO:tasks.cephadm:Adding mds.e on smithi148 2024-04-23T18:33:17.733 INFO:tasks.cephadm:Adding mds.h on smithi148 2024-04-23T18:33:17.733 INFO:tasks.cephadm:Adding mds.k on smithi148 2024-04-23T18:33:17.733 INFO:tasks.cephadm:Adding mds.c on smithi157 2024-04-23T18:33:17.733 INFO:tasks.cephadm:Adding mds.f on smithi157 2024-04-23T18:33:17.733 INFO:tasks.cephadm:Adding mds.i on smithi157 2024-04-23T18:33:17.734 INFO:tasks.cephadm:Adding mds.l on smithi157 2024-04-23T18:33:17.734 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l' 2024-04-23T18:33:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:33:19.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:19 smithi157 ceph-mon[32580]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:19.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:19 smithi148 ceph-mon[32311]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:19.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:19 smithi132 ceph-mon[29178]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:19.614 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.619+0000 7f4e336c9700 1 -- 172.21.15.157:0/412110300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e2c1035c0 msgr2=0x7f4e2c1059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:19.614 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.619+0000 7f4e336c9700 1 --2- 172.21.15.157:0/412110300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e2c1035c0 0x7f4e2c1059b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009990 tx=0x7f4e28009ca0 comp rx=0 tx=0).stop 2024-04-23T18:33:19.614 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.619+0000 7f4e336c9700 1 -- 172.21.15.157:0/412110300 shutdown_connections 2024-04-23T18:33:19.614 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.619+0000 7f4e336c9700 1 --2- 172.21.15.157:0/412110300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c108820 0x7f4e2c10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:19.614 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.619+0000 7f4e336c9700 1 --2- 172.21.15.157:0/412110300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e2c1035c0 0x7f4e2c1059b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.619+0000 7f4e336c9700 1 --2- 172.21.15.157:0/412110300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e2c105ef0 0x7f4e2c1082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e336c9700 1 -- 172.21.15.157:0/412110300 >> 172.21.15.157:0/412110300 conn(0x7f4e2c0fcf70 msgr2=0x7f4e2c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e336c9700 1 -- 172.21.15.157:0/412110300 shutdown_connections 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e336c9700 1 -- 172.21.15.157:0/412110300 wait complete. 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e336c9700 1 Processor -- start 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e336c9700 1 -- start start 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e336c9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e2c1035c0 0x7f4e2c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e336c9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e2c105ef0 0x7f4e2c1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e336c9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c108820 0x7f4e2c1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e336c9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c10e530 con 0x7f4e2c1035c0 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e336c9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c10e3b0 con 0x7f4e2c108820 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e336c9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c10e6b0 con 0x7f4e2c105ef0 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e30c64700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e2c105ef0 0x7f4e2c1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.620+0000 7f4e30c64700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e2c105ef0 0x7f4e2c1990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51360/0 (socket says 172.21.15.157:51360) 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e30c64700 1 -- 172.21.15.157:0/1835007872 learned_addr learned my addr 172.21.15.157:0/1835007872 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e31c66700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c108820 0x7f4e2c1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:19.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e31465700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e2c1035c0 0x7f4e2c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:19.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e30c64700 1 -- 172.21.15.157:0/1835007872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c108820 msgr2=0x7f4e2c1a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:19.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e30c64700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c108820 0x7f4e2c1a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:19.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e30c64700 1 -- 172.21.15.157:0/1835007872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e2c1035c0 msgr2=0x7f4e2c198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:19.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e30c64700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e2c1035c0 0x7f4e2c198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:19.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e30c64700 1 -- 172.21.15.157:0/1835007872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c1a07d0 con 0x7f4e2c105ef0 2024-04-23T18:33:19.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e30c64700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e2c105ef0 0x7f4e2c1990b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00c9e0 tx=0x7f4e1c00ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:19.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e227fc700 1 -- 172.21.15.157:0/1835007872 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c007ac0 con 0x7f4e2c105ef0 2024-04-23T18:33:19.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e336c9700 1 -- 172.21.15.157:0/1835007872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c1a0ac0 con 0x7f4e2c105ef0 2024-04-23T18:33:19.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e227fc700 1 -- 172.21.15.157:0/1835007872 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e1c007c60 con 0x7f4e2c105ef0 2024-04-23T18:33:19.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e227fc700 1 -- 172.21.15.157:0/1835007872 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c018660 con 0x7f4e2c105ef0 2024-04-23T18:33:19.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.621+0000 7f4e336c9700 1 -- 172.21.15.157:0/1835007872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c1a1000 con 0x7f4e2c105ef0 2024-04-23T18:33:19.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.622+0000 7f4e336c9700 1 -- 172.21.15.157:0/1835007872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2c069730 con 0x7f4e2c105ef0 2024-04-23T18:33:19.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.624+0000 7f4e227fc700 1 -- 172.21.15.157:0/1835007872 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4e1c00f450 con 0x7f4e2c105ef0 2024-04-23T18:33:19.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.624+0000 7f4e227fc700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4e180a3c40 0x7f4e180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:19.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.625+0000 7f4e227fc700 1 -- 172.21.15.157:0/1835007872 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4e1c0d01b0 con 0x7f4e2c105ef0 2024-04-23T18:33:19.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.625+0000 7f4e31465700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4e180a3c40 0x7f4e180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:19.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.625+0000 7f4e31465700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4e180a3c40 0x7f4e180a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800b5c0 tx=0x7f4e28005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:19.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.628+0000 7f4e227fc700 1 -- 172.21.15.157:0/1835007872 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e1c014030 con 0x7f4e2c105ef0 2024-04-23T18:33:19.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.841+0000 7f4e336c9700 1 -- 172.21.15.157:0/1835007872 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}) -- 0x7f4e2c062f80 con 0x7f4e180a3c40 2024-04-23T18:33:19.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.849+0000 7f4e227fc700 1 -- 172.21.15.157:0/1835007872 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f4e2c062f80 con 0x7f4e180a3c40 2024-04-23T18:33:19.842 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mds.all update... 2024-04-23T18:33:19.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.851+0000 7f4e336c9700 1 -- 172.21.15.157:0/1835007872 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4e180a3c40 msgr2=0x7f4e180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:19.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.851+0000 7f4e336c9700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4e180a3c40 0x7f4e180a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800b5c0 tx=0x7f4e28005fd0 comp rx=0 tx=0).stop 2024-04-23T18:33:19.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.851+0000 7f4e336c9700 1 -- 172.21.15.157:0/1835007872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e2c105ef0 msgr2=0x7f4e2c1990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:19.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.851+0000 7f4e336c9700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e2c105ef0 0x7f4e2c1990b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00c9e0 tx=0x7f4e1c00ccf0 comp rx=0 tx=0).stop 2024-04-23T18:33:19.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.851+0000 7f4e336c9700 1 -- 172.21.15.157:0/1835007872 shutdown_connections 2024-04-23T18:33:19.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.851+0000 7f4e336c9700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c108820 0x7f4e2c1a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:19.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.851+0000 7f4e336c9700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4e180a3c40 0x7f4e180a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:19.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.851+0000 7f4e336c9700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e2c1035c0 0x7f4e2c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:19.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.851+0000 7f4e336c9700 1 --2- 172.21.15.157:0/1835007872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e2c105ef0 0x7f4e2c1990b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:19.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.851+0000 7f4e336c9700 1 -- 172.21.15.157:0/1835007872 >> 172.21.15.157:0/1835007872 conn(0x7f4e2c0fcf70 msgr2=0x7f4e2c103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:19.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.851+0000 7f4e336c9700 1 -- 172.21.15.157:0/1835007872 shutdown_connections 2024-04-23T18:33:19.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-23T18:33:19.851+0000 7f4e336c9700 1 -- 172.21.15.157:0/1835007872 wait complete. 2024-04-23T18:33:20.510 DEBUG:teuthology.orchestra.run.smithi132:mds.a> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.a.service 2024-04-23T18:33:20.513 DEBUG:teuthology.orchestra.run.smithi132:mds.d> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.d.service 2024-04-23T18:33:20.556 DEBUG:teuthology.orchestra.run.smithi132:mds.g> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.g.service 2024-04-23T18:33:20.600 DEBUG:teuthology.orchestra.run.smithi132:mds.j> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.j.service 2024-04-23T18:33:20.643 DEBUG:teuthology.orchestra.run.smithi148:mds.b> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.b.service 2024-04-23T18:33:20.646 DEBUG:teuthology.orchestra.run.smithi148:mds.e> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.e.service 2024-04-23T18:33:20.690 DEBUG:teuthology.orchestra.run.smithi148:mds.h> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.h.service 2024-04-23T18:33:20.733 DEBUG:teuthology.orchestra.run.smithi148:mds.k> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.k.service 2024-04-23T18:33:20.776 DEBUG:teuthology.orchestra.run.smithi157:mds.c> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.c.service 2024-04-23T18:33:20.779 DEBUG:teuthology.orchestra.run.smithi157:mds.f> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.f.service 2024-04-23T18:33:20.822 DEBUG:teuthology.orchestra.run.smithi157:mds.i> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.i.service 2024-04-23T18:33:20.864 DEBUG:teuthology.orchestra.run.smithi157:mds.l> sudo journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.l.service 2024-04-23T18:33:20.908 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-23T18:33:20.908 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:33:21.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:20 smithi148 ceph-mon[32311]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:33:21.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:20 smithi148 ceph-mon[32311]: Saving service mds.all spec with placement smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-04-23T18:33:21.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:21.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:21.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:21.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:21.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:21.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:21.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:21.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:21.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:20 smithi148 ceph-mon[32311]: Deploying daemon mds.l on smithi157 2024-04-23T18:33:21.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:20 smithi148 ceph-mon[32311]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:21.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.031+0000 7f02a62a9640 1 -- 172.21.15.132:0/304820564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a0100040 msgr2=0x7f02a01004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.031+0000 7f02a62a9640 1 --2- 172.21.15.132:0/304820564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a0100040 0x7f02a01004c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0294009900 tx=0x7f029402a410 comp rx=0 tx=0).stop 2024-04-23T18:33:21.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.031+0000 7f02a62a9640 1 -- 172.21.15.132:0/304820564 shutdown_connections 2024-04-23T18:33:21.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.031+0000 7f02a62a9640 1 --2- 172.21.15.132:0/304820564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a0100a00 0x7f02a0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.031+0000 7f02a62a9640 1 --2- 172.21.15.132:0/304820564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a0100040 0x7f02a01004c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.031+0000 7f02a62a9640 1 --2- 172.21.15.132:0/304820564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02a01024c0 0x7f02a01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.031+0000 7f02a62a9640 1 -- 172.21.15.132:0/304820564 >> 172.21.15.132:0/304820564 conn(0x7f02a00fbbc0 msgr2=0x7f02a00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:21.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.032+0000 7f02a62a9640 1 -- 172.21.15.132:0/304820564 shutdown_connections 2024-04-23T18:33:21.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.032+0000 7f02a62a9640 1 -- 172.21.15.132:0/304820564 wait complete. 2024-04-23T18:33:21.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.032+0000 7f02a62a9640 1 Processor -- start 2024-04-23T18:33:21.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.032+0000 7f02a62a9640 1 -- start start 2024-04-23T18:33:21.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f02a62a9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02a0100040 0x7f02a019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f02a62a9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a0100a00 0x7f02a019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f02a62a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a01024c0 0x7f02a01a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f02a62a9640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f02a010fd00 con 0x7f02a0100a00 2024-04-23T18:33:21.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f02a62a9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f02a010fb80 con 0x7f02a0100040 2024-04-23T18:33:21.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f02a62a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f02a010fe80 con 0x7f02a01024c0 2024-04-23T18:33:21.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a0100a00 0x7f02a019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a0100a00 0x7f02a019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:38968/0 (socket says 172.21.15.132:38968) 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029f7fe640 1 -- 172.21.15.132:0/1968895748 learned_addr learned my addr 172.21.15.132:0/1968895748 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f02a4846640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a01024c0 0x7f02a01a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029ffff640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02a0100040 0x7f02a019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029f7fe640 1 -- 172.21.15.132:0/1968895748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a01024c0 msgr2=0x7f02a01a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029f7fe640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a01024c0 0x7f02a01a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029f7fe640 1 -- 172.21.15.132:0/1968895748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02a0100040 msgr2=0x7f02a019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029f7fe640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02a0100040 0x7f02a019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029f7fe640 1 -- 172.21.15.132:0/1968895748 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a01a1fe0 con 0x7f02a0100a00 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029ffff640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02a0100040 0x7f02a019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029f7fe640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a0100a00 0x7f02a019a900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0294005940 tx=0x7f0294031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029d7fa640 1 -- 172.21.15.132:0/1968895748 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029403f030 con 0x7f02a0100a00 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029d7fa640 1 -- 172.21.15.132:0/1968895748 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0294003aa0 con 0x7f02a0100a00 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f02a62a9640 1 -- 172.21.15.132:0/1968895748 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a01a2270 con 0x7f02a0100a00 2024-04-23T18:33:21.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f029d7fa640 1 -- 172.21.15.132:0/1968895748 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294035070 con 0x7f02a0100a00 2024-04-23T18:33:21.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.033+0000 7f02a62a9640 1 -- 172.21.15.132:0/1968895748 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a01a2760 con 0x7f02a0100a00 2024-04-23T18:33:21.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.035+0000 7f02a62a9640 1 -- 172.21.15.132:0/1968895748 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a010c090 con 0x7f02a0100a00 2024-04-23T18:33:21.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.036+0000 7f029d7fa640 1 -- 172.21.15.132:0/1968895748 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0294042030 con 0x7f02a0100a00 2024-04-23T18:33:21.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.036+0000 7f029d7fa640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f02880a3bb0 0x7f02880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.036+0000 7f029ffff640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f02880a3bb0 0x7f02880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.036+0000 7f029d7fa640 1 -- 172.21.15.132:0/1968895748 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f02940f05a0 con 0x7f02a0100a00 2024-04-23T18:33:21.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.037+0000 7f029ffff640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f02880a3bb0 0x7f02880a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0290005d00 tx=0x7f0290005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:21.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.041+0000 7f029d7fa640 1 -- 172.21.15.132:0/1968895748 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02940bd420 con 0x7f02a0100a00 2024-04-23T18:33:21.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:20 smithi157 ceph-mon[32580]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:33:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:20 smithi157 ceph-mon[32580]: Saving service mds.all spec with placement smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-04-23T18:33:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:20 smithi157 ceph-mon[32580]: Deploying daemon mds.l on smithi157 2024-04-23T18:33:21.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:20 smithi157 ceph-mon[32580]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:21.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:20 smithi132 ceph-mon[29178]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:33:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:20 smithi132 ceph-mon[29178]: Saving service mds.all spec with placement smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-04-23T18:33:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:20 smithi132 ceph-mon[29178]: Deploying daemon mds.l on smithi157 2024-04-23T18:33:21.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:20 smithi132 ceph-mon[29178]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:21.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.220+0000 7f02a62a9640 1 -- 172.21.15.132:0/1968895748 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02a002d7e0 con 0x7f02a0100a00 2024-04-23T18:33:21.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.221+0000 7f029d7fa640 1 -- 172.21.15.132:0/1968895748 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18538 (secure 0 0 0) 0x7f02940bcd20 con 0x7f02a0100a00 2024-04-23T18:33:21.213 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:21.214 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":82,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:33:11.819374+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:33:21.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.223+0000 7f02a62a9640 1 -- 172.21.15.132:0/1968895748 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f02880a3bb0 msgr2=0x7f02880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.223+0000 7f02a62a9640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f02880a3bb0 0x7f02880a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0290005d00 tx=0x7f0290005ed0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.223+0000 7f02a62a9640 1 -- 172.21.15.132:0/1968895748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a0100a00 msgr2=0x7f02a019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.223+0000 7f02a62a9640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a0100a00 0x7f02a019a900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0294005940 tx=0x7f0294031a50 comp rx=0 tx=0).stop 2024-04-23T18:33:21.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.223+0000 7f02a62a9640 1 -- 172.21.15.132:0/1968895748 shutdown_connections 2024-04-23T18:33:21.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.223+0000 7f02a62a9640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f02880a3bb0 0x7f02880a6070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.223+0000 7f02a62a9640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a01024c0 0x7f02a01a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.223+0000 7f02a62a9640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a0100a00 0x7f02a019a900 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.223+0000 7f02a62a9640 1 --2- 172.21.15.132:0/1968895748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02a0100040 0x7f02a019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.223+0000 7f02a62a9640 1 -- 172.21.15.132:0/1968895748 >> 172.21.15.132:0/1968895748 conn(0x7f02a00fbbc0 msgr2=0x7f02a00fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:21.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.224+0000 7f02a62a9640 1 -- 172.21.15.132:0/1968895748 shutdown_connections 2024-04-23T18:33:21.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.224+0000 7f02a62a9640 1 -- 172.21.15.132:0/1968895748 wait complete. 2024-04-23T18:33:21.227 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:29:20.132451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:33:21.227 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:33:21.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.336+0000 7f27fa56a640 1 -- 172.21.15.132:0/920963683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27f4102040 msgr2=0x7f27f41024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.336+0000 7f27fa56a640 1 --2- 172.21.15.132:0/920963683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27f4102040 0x7f27f41024c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009900 tx=0x7f27e802a410 comp rx=0 tx=0).stop 2024-04-23T18:33:21.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.336+0000 7f27fa56a640 1 -- 172.21.15.132:0/920963683 shutdown_connections 2024-04-23T18:33:21.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.336+0000 7f27fa56a640 1 --2- 172.21.15.132:0/920963683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f4102a00 0x7f27f4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.336+0000 7f27fa56a640 1 --2- 172.21.15.132:0/920963683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27f4102040 0x7f27f41024c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.336+0000 7f27fa56a640 1 --2- 172.21.15.132:0/920963683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27f4100e60 0x7f27f4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.336+0000 7f27fa56a640 1 -- 172.21.15.132:0/920963683 >> 172.21.15.132:0/920963683 conn(0x7f27f40fc440 msgr2=0x7f27f40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:21.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.336+0000 7f27fa56a640 1 -- 172.21.15.132:0/920963683 shutdown_connections 2024-04-23T18:33:21.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.336+0000 7f27fa56a640 1 -- 172.21.15.132:0/920963683 wait complete. 2024-04-23T18:33:21.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.337+0000 7f27fa56a640 1 Processor -- start 2024-04-23T18:33:21.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.337+0000 7f27fa56a640 1 -- start start 2024-04-23T18:33:21.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.337+0000 7f27fa56a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27f4100e60 0x7f27f406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.337+0000 7f27fa56a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f4102040 0x7f27f419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.337+0000 7f27fa56a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27f4102a00 0x7f27f419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.337+0000 7f27fa56a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f27f410ca40 con 0x7f27f4100e60 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.337+0000 7f27fa56a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27f410c8c0 con 0x7f27f4102a00 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.337+0000 7f27fa56a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27f410cbc0 con 0x7f27f4102040 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.337+0000 7f27f3fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27f4100e60 0x7f27f406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.337+0000 7f27f3fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27f4100e60 0x7f27f406b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:38988/0 (socket says 172.21.15.132:38988) 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.337+0000 7f27f3fff640 1 -- 172.21.15.132:0/191254071 learned_addr learned my addr 172.21.15.132:0/191254071 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.337+0000 7f27f37fe640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f4102040 0x7f27f419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.338+0000 7f27f3fff640 1 -- 172.21.15.132:0/191254071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f4102040 msgr2=0x7f27f419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.338+0000 7f27f3fff640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f4102040 0x7f27f419cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.338+0000 7f27f3fff640 1 -- 172.21.15.132:0/191254071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27f4102a00 msgr2=0x7f27f419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.338+0000 7f27f3fff640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27f4102a00 0x7f27f419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.338+0000 7f27f3fff640 1 -- 172.21.15.132:0/191254071 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f419fe80 con 0x7f27f4100e60 2024-04-23T18:33:21.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.338+0000 7f27f3fff640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27f4100e60 0x7f27f406b0c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27e400c970 tx=0x7f27e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:21.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.338+0000 7f27f17fa640 1 -- 172.21.15.132:0/191254071 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4004220 con 0x7f27f4100e60 2024-04-23T18:33:21.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.338+0000 7f27fa56a640 1 -- 172.21.15.132:0/191254071 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f41a0110 con 0x7f27f4100e60 2024-04-23T18:33:21.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.338+0000 7f27f17fa640 1 -- 172.21.15.132:0/191254071 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27e40043c0 con 0x7f27f4100e60 2024-04-23T18:33:21.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.338+0000 7f27f17fa640 1 -- 172.21.15.132:0/191254071 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e400f450 con 0x7f27f4100e60 2024-04-23T18:33:21.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.338+0000 7f27fa56a640 1 -- 172.21.15.132:0/191254071 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f41a0630 con 0x7f27f4100e60 2024-04-23T18:33:21.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.339+0000 7f27fa56a640 1 -- 172.21.15.132:0/191254071 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f41082d0 con 0x7f27f4100e60 2024-04-23T18:33:21.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.340+0000 7f27f17fa640 1 -- 172.21.15.132:0/191254071 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f27e4003610 con 0x7f27f4100e60 2024-04-23T18:33:21.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.341+0000 7f27f17fa640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f27dc0a3bb0 0x7f27dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.341+0000 7f27f37fe640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f27dc0a3bb0 0x7f27dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.341+0000 7f27f17fa640 1 -- 172.21.15.132:0/191254071 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f27e40d01d0 con 0x7f27f4100e60 2024-04-23T18:33:21.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.341+0000 7f27f37fe640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f27dc0a3bb0 0x7f27dc0a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27e8031500 tx=0x7f27e8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:21.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.345+0000 7f27f17fa640 1 -- 172.21.15.132:0/191254071 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27e4014030 con 0x7f27f4100e60 2024-04-23T18:33:21.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.520+0000 7f27fa56a640 1 -- 172.21.15.132:0/191254071 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f27f402d830 con 0x7f27f4100e60 2024-04-23T18:33:21.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.520+0000 7f27f17fa640 1 -- 172.21.15.132:0/191254071 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f27e409cfb0 con 0x7f27f4100e60 2024-04-23T18:33:21.513 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-04-23T18:33:21.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.522+0000 7f27fa56a640 1 -- 172.21.15.132:0/191254071 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f27dc0a3bb0 msgr2=0x7f27dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.522+0000 7f27fa56a640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f27dc0a3bb0 0x7f27dc0a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27e8031500 tx=0x7f27e8005a20 comp rx=0 tx=0).stop 2024-04-23T18:33:21.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.523+0000 7f27fa56a640 1 -- 172.21.15.132:0/191254071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27f4100e60 msgr2=0x7f27f406b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.523+0000 7f27fa56a640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27f4100e60 0x7f27f406b0c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27e400c970 tx=0x7f27e400ce30 comp rx=0 tx=0).stop 2024-04-23T18:33:21.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.523+0000 7f27fa56a640 1 -- 172.21.15.132:0/191254071 shutdown_connections 2024-04-23T18:33:21.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.523+0000 7f27fa56a640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f27dc0a3bb0 0x7f27dc0a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.523+0000 7f27fa56a640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f4102040 0x7f27f419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.523+0000 7f27fa56a640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27f4102a00 0x7f27f419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.523+0000 7f27fa56a640 1 --2- 172.21.15.132:0/191254071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27f4100e60 0x7f27f406b0c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.523+0000 7f27fa56a640 1 -- 172.21.15.132:0/191254071 >> 172.21.15.132:0/191254071 conn(0x7f27f40fc440 msgr2=0x7f27f41054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:21.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.523+0000 7f27fa56a640 1 -- 172.21.15.132:0/191254071 shutdown_connections 2024-04-23T18:33:21.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.523+0000 7f27fa56a640 1 -- 172.21.15.132:0/191254071 wait complete. 2024-04-23T18:33:21.526 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:21.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.632+0000 7f8e77810640 1 -- 172.21.15.132:0/447803471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e70100e90 msgr2=0x7f8e701012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.632+0000 7f8e77810640 1 --2- 172.21.15.132:0/447803471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e70100e90 0x7f8e701012b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009900 tx=0x7f8e6c02a410 comp rx=0 tx=0).stop 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.632+0000 7f8e77810640 1 -- 172.21.15.132:0/447803471 shutdown_connections 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.632+0000 7f8e77810640 1 --2- 172.21.15.132:0/447803471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e70101fe0 0x7f8e70102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.632+0000 7f8e77810640 1 --2- 172.21.15.132:0/447803471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e701029a0 0x7f8e701092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.632+0000 7f8e77810640 1 --2- 172.21.15.132:0/447803471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e70100e90 0x7f8e701012b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.632+0000 7f8e77810640 1 -- 172.21.15.132:0/447803471 >> 172.21.15.132:0/447803471 conn(0x7f8e700fc400 msgr2=0x7f8e700fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.632+0000 7f8e77810640 1 -- 172.21.15.132:0/447803471 shutdown_connections 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.632+0000 7f8e77810640 1 -- 172.21.15.132:0/447803471 wait complete. 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e77810640 1 Processor -- start 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e77810640 1 -- start start 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e77810640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e70100e90 0x7f8e701981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e77810640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e70101fe0 0x7f8e70198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e77810640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e701029a0 0x7f8e7019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e77810640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010c9e0 con 0x7f8e70100e90 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e77810640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010c860 con 0x7f8e70101fe0 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e77810640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010cb60 con 0x7f8e701029a0 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e755ac640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e70100e90 0x7f8e701981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e755ac640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e70100e90 0x7f8e701981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39008/0 (socket says 172.21.15.132:39008) 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e755ac640 1 -- 172.21.15.132:0/3116809000 learned_addr learned my addr 172.21.15.132:0/3116809000 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e75dad640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e701029a0 0x7f8e7019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e74dab640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e70101fe0 0x7f8e70198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e755ac640 1 -- 172.21.15.132:0/3116809000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e701029a0 msgr2=0x7f8e7019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e755ac640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e701029a0 0x7f8e7019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e755ac640 1 -- 172.21.15.132:0/3116809000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e70101fe0 msgr2=0x7f8e70198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e755ac640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e70101fe0 0x7f8e70198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e755ac640 1 -- 172.21.15.132:0/3116809000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7019fe00 con 0x7f8e70100e90 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e74dab640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e70101fe0 0x7f8e70198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e755ac640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e70100e90 0x7f8e701981e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c005940 tx=0x7f8e6c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:21.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e667fc640 1 -- 172.21.15.132:0/3116809000 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c03f030 con 0x7f8e70100e90 2024-04-23T18:33:21.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e667fc640 1 -- 172.21.15.132:0/3116809000 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e6c003aa0 con 0x7f8e70100e90 2024-04-23T18:33:21.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e77810640 1 -- 172.21.15.132:0/3116809000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e701a0090 con 0x7f8e70100e90 2024-04-23T18:33:21.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.633+0000 7f8e667fc640 1 -- 172.21.15.132:0/3116809000 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c035070 con 0x7f8e70100e90 2024-04-23T18:33:21.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.634+0000 7f8e77810640 1 -- 172.21.15.132:0/3116809000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e701a0580 con 0x7f8e70100e90 2024-04-23T18:33:21.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.635+0000 7f8e77810640 1 -- 172.21.15.132:0/3116809000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e701a07e0 con 0x7f8e70100e90 2024-04-23T18:33:21.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.636+0000 7f8e667fc640 1 -- 172.21.15.132:0/3116809000 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8e6c042030 con 0x7f8e70100e90 2024-04-23T18:33:21.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.636+0000 7f8e667fc640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8e5c0a3bb0 0x7f8e5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.636+0000 7f8e74dab640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8e5c0a3bb0 0x7f8e5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.636+0000 7f8e667fc640 1 -- 172.21.15.132:0/3116809000 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8e6c0f05a0 con 0x7f8e70100e90 2024-04-23T18:33:21.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.637+0000 7f8e74dab640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8e5c0a3bb0 0x7f8e5c0a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8e701996f0 tx=0x7f8e60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:21.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.641+0000 7f8e667fc640 1 -- 172.21.15.132:0/3116809000 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e6c0bd420 con 0x7f8e70100e90 2024-04-23T18:33:21.849 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.857+0000 7f8e77810640 1 -- 172.21.15.132:0/3116809000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e701a09b0 con 0x7f8e70100e90 2024-04-23T18:33:21.849 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.857+0000 7f8e667fc640 1 -- 172.21.15.132:0/3116809000 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f8e6c0bcd20 con 0x7f8e70100e90 2024-04-23T18:33:21.850 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:21.850 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-23T18:33:21.850 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 1 2024-04-23T18:33:21.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.859+0000 7f8e77810640 1 -- 172.21.15.132:0/3116809000 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8e5c0a3bb0 msgr2=0x7f8e5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.859+0000 7f8e77810640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8e5c0a3bb0 0x7f8e5c0a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8e701996f0 tx=0x7f8e60005f90 comp rx=0 tx=0).stop 2024-04-23T18:33:21.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.859+0000 7f8e77810640 1 -- 172.21.15.132:0/3116809000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e70100e90 msgr2=0x7f8e701981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.859+0000 7f8e77810640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e70100e90 0x7f8e701981e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c005940 tx=0x7f8e6c031a50 comp rx=0 tx=0).stop 2024-04-23T18:33:21.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.859+0000 7f8e77810640 1 -- 172.21.15.132:0/3116809000 shutdown_connections 2024-04-23T18:33:21.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.859+0000 7f8e77810640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8e5c0a3bb0 0x7f8e5c0a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.859+0000 7f8e77810640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e701029a0 0x7f8e7019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.859+0000 7f8e77810640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e70101fe0 0x7f8e70198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.859+0000 7f8e77810640 1 --2- 172.21.15.132:0/3116809000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e70100e90 0x7f8e701981e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.859+0000 7f8e77810640 1 -- 172.21.15.132:0/3116809000 >> 172.21.15.132:0/3116809000 conn(0x7f8e700fc400 msgr2=0x7f8e700690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:21.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.860+0000 7f8e77810640 1 -- 172.21.15.132:0/3116809000 shutdown_connections 2024-04-23T18:33:21.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:21.860+0000 7f8e77810640 1 -- 172.21.15.132:0/3116809000 wait complete. 2024-04-23T18:33:22.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:21 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1968895748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:33:22.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:21 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/191254071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:33:22.036 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:21 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3116809000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:22.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:21 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1968895748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:33:22.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:21 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/191254071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:33:22.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:21 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3116809000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:22.289 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:33:22 smithi157 systemd[1]: Starting Ceph mds.l for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:33:22.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:21 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1968895748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:33:22.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:21 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/191254071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:33:22.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:21 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3116809000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:22.703 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:33:22 smithi157 podman[62543]: 2024-04-23 18:33:22.355237286 +0000 UTC m=+0.027306674 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:33:22.703 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:33:22 smithi157 podman[62543]: 2024-04-23 18:33:22.475934345 +0000 UTC m=+0.148003706 container create d2b89506d0e6d58689ab6bc06f0bdb1b8981fdd049817cd06a220b05419ef853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-23T18:33:22.703 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:33:22 smithi157 podman[62543]: 2024-04-23 18:33:22.711944274 +0000 UTC m=+0.384013646 container init d2b89506d0e6d58689ab6bc06f0bdb1b8981fdd049817cd06a220b05419ef853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:33:23.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:22 smithi157 ceph-mon[32580]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:23.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:22 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] up:boot 2024-04-23T18:33:23.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:22 smithi157 ceph-mon[32580]: fsmap 1 up:standby 2024-04-23T18:33:23.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:22 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:33:23.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:22 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:23.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:22 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:23.040 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:33:22 smithi157 podman[62543]: 2024-04-23 18:33:22.71453055 +0000 UTC m=+0.386599915 container start d2b89506d0e6d58689ab6bc06f0bdb1b8981fdd049817cd06a220b05419ef853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-23T18:33:23.041 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:33:22 smithi157 ceph-mds[62562]: -- 172.21.15.157:0/1435847041 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0c754ba40 con 0x55e0c751f000 2024-04-23T18:33:23.041 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:33:22 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l[62558]: starting mds.l at 2024-04-23T18:33:23.041 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:33:22 smithi157 bash[62543]: d2b89506d0e6d58689ab6bc06f0bdb1b8981fdd049817cd06a220b05419ef853 2024-04-23T18:33:23.041 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:33:22 smithi157 systemd[1]: Started Ceph mds.l for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:33:23.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:22 smithi148 ceph-mon[32311]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:23.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:22 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] up:boot 2024-04-23T18:33:23.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:22 smithi148 ceph-mon[32311]: fsmap 1 up:standby 2024-04-23T18:33:23.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:22 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:33:23.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:22 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:23.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:22 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:23.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:22 smithi132 ceph-mon[29178]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:23.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:22 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] up:boot 2024-04-23T18:33:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:22 smithi132 ceph-mon[29178]: fsmap 1 up:standby 2024-04-23T18:33:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:22 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:33:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:22 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:23.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:22 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:23.865 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:23.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.983+0000 7f3257d08640 1 -- 172.21.15.132:0/3260039388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3250102a30 msgr2=0x7f3250104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:23.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.983+0000 7f3257d08640 1 --2- 172.21.15.132:0/3260039388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3250102a30 0x7f3250104e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f324c009900 tx=0x7f324c02a410 comp rx=0 tx=0).stop 2024-04-23T18:33:23.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.983+0000 7f3257d08640 1 -- 172.21.15.132:0/3260039388 shutdown_connections 2024-04-23T18:33:23.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.983+0000 7f3257d08640 1 --2- 172.21.15.132:0/3260039388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3250105360 0x7f3250107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:23.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.983+0000 7f3257d08640 1 --2- 172.21.15.132:0/3260039388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3250107c90 0x7f325010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:23.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.983+0000 7f3257d08640 1 --2- 172.21.15.132:0/3260039388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3250102a30 0x7f3250104e20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:23.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.983+0000 7f3257d08640 1 -- 172.21.15.132:0/3260039388 >> 172.21.15.132:0/3260039388 conn(0x7f32500fc440 msgr2=0x7f32500fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:23.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.983+0000 7f3257d08640 1 -- 172.21.15.132:0/3260039388 shutdown_connections 2024-04-23T18:33:23.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.983+0000 7f3257d08640 1 -- 172.21.15.132:0/3260039388 wait complete. 2024-04-23T18:33:23.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f3257d08640 1 Processor -- start 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f3257d08640 1 -- start start 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f3257d08640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3250102a30 0x7f3250198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f3257d08640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3250105360 0x7f32501987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f3257d08640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3250107c90 0x7f325019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f3257d08640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f325010daf0 con 0x7f3250107c90 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f3257d08640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f325010d970 con 0x7f3250102a30 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f3257d08640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f325010dc70 con 0x7f3250105360 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f32562a5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3250107c90 0x7f325019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f32562a5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3250107c90 0x7f325019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39038/0 (socket says 172.21.15.132:39038) 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f32562a5640 1 -- 172.21.15.132:0/1298383663 learned_addr learned my addr 172.21.15.132:0/1298383663 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f3255aa4640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3250102a30 0x7f3250198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f32552a3640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3250105360 0x7f32501987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f32562a5640 1 -- 172.21.15.132:0/1298383663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3250105360 msgr2=0x7f32501987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:23.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f32562a5640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3250105360 0x7f32501987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:23.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f32562a5640 1 -- 172.21.15.132:0/1298383663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3250102a30 msgr2=0x7f3250198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:23.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.984+0000 7f32562a5640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3250102a30 0x7f3250198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:23.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.985+0000 7f32562a5640 1 -- 172.21.15.132:0/1298383663 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325019fe80 con 0x7f3250107c90 2024-04-23T18:33:23.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.985+0000 7f32552a3640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3250105360 0x7f32501987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:23.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.985+0000 7f32562a5640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3250107c90 0x7f325019f810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3248009fd0 tx=0x7f324800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:23.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.985+0000 7f3246ffd640 1 -- 172.21.15.132:0/1298383663 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248017070 con 0x7f3250107c90 2024-04-23T18:33:23.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.985+0000 7f3257d08640 1 -- 172.21.15.132:0/1298383663 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32501a0170 con 0x7f3250107c90 2024-04-23T18:33:23.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.985+0000 7f3246ffd640 1 -- 172.21.15.132:0/1298383663 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3248003e20 con 0x7f3250107c90 2024-04-23T18:33:23.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.985+0000 7f3246ffd640 1 -- 172.21.15.132:0/1298383663 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248010460 con 0x7f3250107c90 2024-04-23T18:33:23.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.985+0000 7f3257d08640 1 -- 172.21.15.132:0/1298383663 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32501a06c0 con 0x7f3250107c90 2024-04-23T18:33:23.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.986+0000 7f3257d08640 1 -- 172.21.15.132:0/1298383663 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32500fdf30 con 0x7f3250107c90 2024-04-23T18:33:23.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.987+0000 7f3246ffd640 1 -- 172.21.15.132:0/1298383663 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f3248003970 con 0x7f3250107c90 2024-04-23T18:33:23.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.987+0000 7f3246ffd640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f323c0a3bb0 0x7f323c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:23.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.988+0000 7f3255aa4640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f323c0a3bb0 0x7f323c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:23.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.988+0000 7f3246ffd640 1 -- 172.21.15.132:0/1298383663 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f32480cfd20 con 0x7f3250107c90 2024-04-23T18:33:23.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.988+0000 7f3255aa4640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f323c0a3bb0 0x7f323c0a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f324c000c00 tx=0x7f324c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:23.984 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:23.992+0000 7f3246ffd640 1 -- 172.21.15.132:0/1298383663 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f324809cba0 con 0x7f3250107c90 2024-04-23T18:33:24.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.208+0000 7f3257d08640 1 -- 172.21.15.132:0/1298383663 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3250062b50 con 0x7f3250107c90 2024-04-23T18:33:24.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.208+0000 7f3246ffd640 1 -- 172.21.15.132:0/1298383663 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f324809c4a0 con 0x7f3250107c90 2024-04-23T18:33:24.201 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:24.201 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T18:33:24.201 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 2 2024-04-23T18:33:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.210+0000 7f3257d08640 1 -- 172.21.15.132:0/1298383663 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f323c0a3bb0 msgr2=0x7f323c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.210+0000 7f3257d08640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f323c0a3bb0 0x7f323c0a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f324c000c00 tx=0x7f324c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.210+0000 7f3257d08640 1 -- 172.21.15.132:0/1298383663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3250107c90 msgr2=0x7f325019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.210+0000 7f3257d08640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3250107c90 0x7f325019f810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3248009fd0 tx=0x7f324800b260 comp rx=0 tx=0).stop 2024-04-23T18:33:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.210+0000 7f3257d08640 1 -- 172.21.15.132:0/1298383663 shutdown_connections 2024-04-23T18:33:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.210+0000 7f3257d08640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f323c0a3bb0 0x7f323c0a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.210+0000 7f3257d08640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3250105360 0x7f32501987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.210+0000 7f3257d08640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3250107c90 0x7f325019f810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.210+0000 7f3257d08640 1 --2- 172.21.15.132:0/1298383663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3250102a30 0x7f3250198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:24.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.210+0000 7f3257d08640 1 -- 172.21.15.132:0/1298383663 >> 172.21.15.132:0/1298383663 conn(0x7f32500fc440 msgr2=0x7f3250103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:24.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.211+0000 7f3257d08640 1 -- 172.21.15.132:0/1298383663 shutdown_connections 2024-04-23T18:33:24.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:24.211+0000 7f3257d08640 1 -- 172.21.15.132:0/1298383663 wait complete. 2024-04-23T18:33:24.213 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:24.213 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:24.213 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:24.213 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:23 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:24.213 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:23 smithi132 ceph-mon[29178]: Deploying daemon mds.k on smithi148 2024-04-23T18:33:24.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:24.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:24.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:24.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:23 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:24.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:23 smithi148 ceph-mon[32311]: Deploying daemon mds.k on smithi148 2024-04-23T18:33:24.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:24.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:24.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:24.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:23 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:24.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:23 smithi157 ceph-mon[32580]: Deploying daemon mds.k on smithi148 2024-04-23T18:33:25.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:24 smithi148 ceph-mon[32311]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:25.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:24 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1298383663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:25.265 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:33:25 smithi148 systemd[1]: Starting Ceph mds.k for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:33:25.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:24 smithi157 ceph-mon[32580]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:25.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:24 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1298383663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:25.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:24 smithi132 ceph-mon[29178]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:25.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:24 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1298383663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:25.598 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:33:25 smithi148 podman[61573]: 2024-04-23 18:33:25.233966351 +0000 UTC m=+0.018011012 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:33:25.598 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:33:25 smithi148 podman[61573]: 2024-04-23 18:33:25.353244119 +0000 UTC m=+0.137288830 container create 1c7c47ae3cda81e321a523b47d12ffba6ca810338ca8d49f99a84bde86e65544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True) 2024-04-23T18:33:26.014 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:33:25 smithi148 podman[61573]: 2024-04-23 18:33:25.606336829 +0000 UTC m=+0.390381481 container init 1c7c47ae3cda81e321a523b47d12ffba6ca810338ca8d49f99a84bde86e65544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:33:26.015 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:33:25 smithi148 podman[61573]: 2024-04-23 18:33:25.609593836 +0000 UTC m=+0.393638479 container start 1c7c47ae3cda81e321a523b47d12ffba6ca810338ca8d49f99a84bde86e65544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-23T18:33:26.015 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:33:25 smithi148 ceph-mds[61592]: -- 172.21.15.148:0/785348288 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bcfdacba40 con 0x55bcfda9f000 2024-04-23T18:33:26.015 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:33:25 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-k[61588]: starting mds.k at 2024-04-23T18:33:26.015 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:33:25 smithi148 bash[61573]: 1c7c47ae3cda81e321a523b47d12ffba6ca810338ca8d49f99a84bde86e65544 2024-04-23T18:33:26.015 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:33:25 smithi148 systemd[1]: Started Ceph mds.k for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:33:26.214 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:26.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.334+0000 7fa1f5e13640 1 -- 172.21.15.132:0/2353272075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f0101fe0 msgr2=0x7fa1f0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:26.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.334+0000 7fa1f5e13640 1 --2- 172.21.15.132:0/2353272075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f0101fe0 0x7fa1f0102460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009900 tx=0x7fa1e402a410 comp rx=0 tx=0).stop 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.334+0000 7fa1f5e13640 1 -- 172.21.15.132:0/2353272075 shutdown_connections 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.334+0000 7fa1f5e13640 1 --2- 172.21.15.132:0/2353272075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1f01029a0 0x7fa1f01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.334+0000 7fa1f5e13640 1 --2- 172.21.15.132:0/2353272075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f0101fe0 0x7fa1f0102460 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.334+0000 7fa1f5e13640 1 --2- 172.21.15.132:0/2353272075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f0100e90 0x7fa1f01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.334+0000 7fa1f5e13640 1 -- 172.21.15.132:0/2353272075 >> 172.21.15.132:0/2353272075 conn(0x7fa1f00fc400 msgr2=0x7fa1f00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.334+0000 7fa1f5e13640 1 -- 172.21.15.132:0/2353272075 shutdown_connections 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.334+0000 7fa1f5e13640 1 -- 172.21.15.132:0/2353272075 wait complete. 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.334+0000 7fa1f5e13640 1 Processor -- start 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1f5e13640 1 -- start start 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1f5e13640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f0100e90 0x7fa1f0104dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1f5e13640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f0101fe0 0x7fa1f0103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1f5e13640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1f01029a0 0x7fa1f0103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1f5e13640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f010ca30 con 0x7fa1f0101fe0 2024-04-23T18:33:26.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1f5e13640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f010c8b0 con 0x7fa1f0100e90 2024-04-23T18:33:26.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1f5e13640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f010cbb0 con 0x7fa1f01029a0 2024-04-23T18:33:26.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1eeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f0101fe0 0x7fa1f0103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:26.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1eeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f0101fe0 0x7fa1f0103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39064/0 (socket says 172.21.15.132:39064) 2024-04-23T18:33:26.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1eeffd640 1 -- 172.21.15.132:0/4148377386 learned_addr learned my addr 172.21.15.132:0/4148377386 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:26.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1ef7fe640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f0100e90 0x7fa1f0104dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:26.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1effff640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1f01029a0 0x7fa1f0103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:26.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1eeffd640 1 -- 172.21.15.132:0/4148377386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1f01029a0 msgr2=0x7fa1f0103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:26.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1eeffd640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1f01029a0 0x7fa1f0103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:26.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1eeffd640 1 -- 172.21.15.132:0/4148377386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f0100e90 msgr2=0x7fa1f0104dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:26.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1eeffd640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f0100e90 0x7fa1f0104dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:26.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1eeffd640 1 -- 172.21.15.132:0/4148377386 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f0104250 con 0x7fa1f0101fe0 2024-04-23T18:33:26.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1ef7fe640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f0100e90 0x7fa1f0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:26.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1effff640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1f01029a0 0x7fa1f0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:26.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.335+0000 7fa1eeffd640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f0101fe0 0x7fa1f0103450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4031500 tx=0x7fa1e4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:26.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.336+0000 7fa1ecff9640 1 -- 172.21.15.132:0/4148377386 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e403f070 con 0x7fa1f0101fe0 2024-04-23T18:33:26.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.336+0000 7fa1ecff9640 1 -- 172.21.15.132:0/4148377386 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1e4031c50 con 0x7fa1f0101fe0 2024-04-23T18:33:26.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.336+0000 7fa1ecff9640 1 -- 172.21.15.132:0/4148377386 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4035030 con 0x7fa1f0101fe0 2024-04-23T18:33:26.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.336+0000 7fa1f5e13640 1 -- 172.21.15.132:0/4148377386 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f01a22f0 con 0x7fa1f0101fe0 2024-04-23T18:33:26.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.336+0000 7fa1f5e13640 1 -- 172.21.15.132:0/4148377386 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f01a2760 con 0x7fa1f0101fe0 2024-04-23T18:33:26.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.337+0000 7fa1f5e13640 1 -- 172.21.15.132:0/4148377386 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f01a29c0 con 0x7fa1f0101fe0 2024-04-23T18:33:26.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.339+0000 7fa1ecff9640 1 -- 172.21.15.132:0/4148377386 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa1e4042070 con 0x7fa1f0101fe0 2024-04-23T18:33:26.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.340+0000 7fa1ecff9640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa1d80a3c30 0x7fa1d80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:26.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.340+0000 7fa1ef7fe640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa1d80a3c30 0x7fa1d80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:26.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.340+0000 7fa1ecff9640 1 -- 172.21.15.132:0/4148377386 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa1e40f0d90 con 0x7fa1f0101fe0 2024-04-23T18:33:26.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.340+0000 7fa1ef7fe640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa1d80a3c30 0x7fa1d80a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009fc0 tx=0x7fa1e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:26.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.344+0000 7fa1ecff9640 1 -- 172.21.15.132:0/4148377386 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1e40bdc10 con 0x7fa1f0101fe0 2024-04-23T18:33:26.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.573+0000 7fa1f5e13640 1 -- 172.21.15.132:0/4148377386 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1f01a2b90 con 0x7fa1f0101fe0 2024-04-23T18:33:26.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.573+0000 7fa1ecff9640 1 -- 172.21.15.132:0/4148377386 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fa1e40bd510 con 0x7fa1f0101fe0 2024-04-23T18:33:26.566 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:26.566 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T18:33:26.566 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 3 2024-04-23T18:33:26.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.575+0000 7fa1f5e13640 1 -- 172.21.15.132:0/4148377386 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa1d80a3c30 msgr2=0x7fa1d80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:26.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.575+0000 7fa1f5e13640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa1d80a3c30 0x7fa1d80a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009fc0 tx=0x7fa1e0005f90 comp rx=0 tx=0).stop 2024-04-23T18:33:26.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.575+0000 7fa1f5e13640 1 -- 172.21.15.132:0/4148377386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f0101fe0 msgr2=0x7fa1f0103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:26.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.575+0000 7fa1f5e13640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f0101fe0 0x7fa1f0103450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4031500 tx=0x7fa1e4003c10 comp rx=0 tx=0).stop 2024-04-23T18:33:26.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.575+0000 7fa1f5e13640 1 -- 172.21.15.132:0/4148377386 shutdown_connections 2024-04-23T18:33:26.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.575+0000 7fa1f5e13640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa1d80a3c30 0x7fa1d80a60f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:26.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.575+0000 7fa1f5e13640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1f01029a0 0x7fa1f0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:26.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.575+0000 7fa1f5e13640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f0101fe0 0x7fa1f0103450 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:26.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.575+0000 7fa1f5e13640 1 --2- 172.21.15.132:0/4148377386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f0100e90 0x7fa1f0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:26.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.575+0000 7fa1f5e13640 1 -- 172.21.15.132:0/4148377386 >> 172.21.15.132:0/4148377386 conn(0x7fa1f00fc400 msgr2=0x7fa1f00fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:26.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.575+0000 7fa1f5e13640 1 -- 172.21.15.132:0/4148377386 shutdown_connections 2024-04-23T18:33:26.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:26.575+0000 7fa1f5e13640 1 -- 172.21.15.132:0/4148377386 wait complete. 2024-04-23T18:33:26.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:26.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:26.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:26.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:26.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:26.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:26.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:26 smithi132 ceph-mon[29178]: Deploying daemon mds.j on smithi132 2024-04-23T18:33:26.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:26 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] up:boot 2024-04-23T18:33:26.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:26 smithi132 ceph-mon[29178]: fsmap 2 up:standby 2024-04-23T18:33:26.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:26 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:33:26.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:26 smithi132 ceph-mon[29178]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:26.863 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:26 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/4148377386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:27.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:26 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:27.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:26 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:27.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:26 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:27.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:26 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:27.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:26 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:27.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:26 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:27.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:26 smithi148 ceph-mon[32311]: Deploying daemon mds.j on smithi132 2024-04-23T18:33:27.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:26 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] up:boot 2024-04-23T18:33:27.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:26 smithi148 ceph-mon[32311]: fsmap 2 up:standby 2024-04-23T18:33:27.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:26 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:33:27.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:26 smithi148 ceph-mon[32311]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:27.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:26 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/4148377386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:27.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:26 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:27.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:26 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:27.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:26 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:27.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:26 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:27.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:26 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:27.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:26 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:27.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:26 smithi157 ceph-mon[32580]: Deploying daemon mds.j on smithi132 2024-04-23T18:33:27.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:26 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] up:boot 2024-04-23T18:33:27.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:26 smithi157 ceph-mon[32580]: fsmap 2 up:standby 2024-04-23T18:33:27.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:26 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:33:27.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:26 smithi157 ceph-mon[32580]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:27.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:26 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/4148377386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:28.112 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:33:28 smithi132 systemd[1]: Starting Ceph mds.j for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:33:28.490 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:33:28 smithi132 podman[69380]: 2024-04-23 18:33:28.134233495 +0000 UTC m=+0.018377633 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:33:28.490 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:33:28 smithi132 podman[69380]: 2024-04-23 18:33:28.262378 +0000 UTC m=+0.146522140 container create 59874b8d89d27ac8fa3af79c0b45ea12c54619e5e595dffd1a4457be4c5b92b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-23T18:33:28.490 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:33:28 smithi132 podman[69380]: 2024-04-23 18:33:28.49861943 +0000 UTC m=+0.382763568 container init 59874b8d89d27ac8fa3af79c0b45ea12c54619e5e595dffd1a4457be4c5b92b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:33:28.580 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:28.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.696+0000 7f2f53a43640 1 -- 172.21.15.132:0/3224757887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c100880 msgr2=0x7f2f4c100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:28.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.696+0000 7f2f53a43640 1 --2- 172.21.15.132:0/3224757887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c100880 0x7f2f4c100ca0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009900 tx=0x7f2f4802a480 comp rx=0 tx=0).stop 2024-04-23T18:33:28.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.696+0000 7f2f53a43640 1 -- 172.21.15.132:0/3224757887 shutdown_connections 2024-04-23T18:33:28.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.696+0000 7f2f53a43640 1 --2- 172.21.15.132:0/3224757887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f4c1011e0 0x7f2f4c101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:28.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.696+0000 7f2f53a43640 1 --2- 172.21.15.132:0/3224757887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f4c101b80 0x7f2f4c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:28.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.696+0000 7f2f53a43640 1 --2- 172.21.15.132:0/3224757887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c100880 0x7f2f4c100ca0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:28.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.696+0000 7f2f53a43640 1 -- 172.21.15.132:0/3224757887 >> 172.21.15.132:0/3224757887 conn(0x7f2f4c0fc400 msgr2=0x7f2f4c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:28.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.696+0000 7f2f53a43640 1 -- 172.21.15.132:0/3224757887 shutdown_connections 2024-04-23T18:33:28.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.696+0000 7f2f53a43640 1 -- 172.21.15.132:0/3224757887 wait complete. 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f53a43640 1 Processor -- start 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f53a43640 1 -- start start 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f53a43640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f4c100880 0x7f2f4c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f53a43640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c1011e0 0x7f2f4c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f53a43640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f4c101b80 0x7f2f4c1a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f53a43640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c10fcf0 con 0x7f2f4c1011e0 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f53a43640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c10fb70 con 0x7f2f4c101b80 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f53a43640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c10fe70 con 0x7f2f4c100880 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f50fde640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c1011e0 0x7f2f4c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f50fde640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c1011e0 0x7f2f4c19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39102/0 (socket says 172.21.15.132:39102) 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f50fde640 1 -- 172.21.15.132:0/2448873714 learned_addr learned my addr 172.21.15.132:0/2448873714 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f51fe0640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f4c101b80 0x7f2f4c1a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f517df640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f4c100880 0x7f2f4c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f50fde640 1 -- 172.21.15.132:0/2448873714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f4c100880 msgr2=0x7f2f4c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:28.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f50fde640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f4c100880 0x7f2f4c19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:28.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f50fde640 1 -- 172.21.15.132:0/2448873714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f4c101b80 msgr2=0x7f2f4c1a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:28.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f50fde640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f4c101b80 0x7f2f4c1a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:28.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f50fde640 1 -- 172.21.15.132:0/2448873714 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4c1a2010 con 0x7f2f4c1011e0 2024-04-23T18:33:28.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f50fde640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c1011e0 0x7f2f4c19a930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00d950 tx=0x7f2f3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:28.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f427fc640 1 -- 172.21.15.132:0/2448873714 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c017070 con 0x7f2f4c1011e0 2024-04-23T18:33:28.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f53a43640 1 -- 172.21.15.132:0/2448873714 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4c1a2300 con 0x7f2f4c1011e0 2024-04-23T18:33:28.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f53a43640 1 -- 172.21.15.132:0/2448873714 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4c1a28a0 con 0x7f2f4c1011e0 2024-04-23T18:33:28.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f427fc640 1 -- 172.21.15.132:0/2448873714 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f3c009c30 con 0x7f2f4c1011e0 2024-04-23T18:33:28.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.697+0000 7f2f427fc640 1 -- 172.21.15.132:0/2448873714 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00f620 con 0x7f2f4c1011e0 2024-04-23T18:33:28.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.698+0000 7f2f53a43640 1 -- 172.21.15.132:0/2448873714 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f4c10b7a0 con 0x7f2f4c1011e0 2024-04-23T18:33:28.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.703+0000 7f2f427fc640 1 -- 172.21.15.132:0/2448873714 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2f3c010460 con 0x7f2f4c1011e0 2024-04-23T18:33:28.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.703+0000 7f2f427fc640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2f380a3bb0 0x7f2f380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:28.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.703+0000 7f2f517df640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2f380a3bb0 0x7f2f380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:28.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.703+0000 7f2f427fc640 1 -- 172.21.15.132:0/2448873714 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2f3c0cfb70 con 0x7f2f4c1011e0 2024-04-23T18:33:28.696 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.704+0000 7f2f517df640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2f380a3bb0 0x7f2f380a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2f48031500 tx=0x7f2f48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:28.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.707+0000 7f2f427fc640 1 -- 172.21.15.132:0/2448873714 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f3c09c9f0 con 0x7f2f4c1011e0 2024-04-23T18:33:28.861 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:33:28 smithi132 podman[69380]: 2024-04-23 18:33:28.501340795 +0000 UTC m=+0.385484941 container start 59874b8d89d27ac8fa3af79c0b45ea12c54619e5e595dffd1a4457be4c5b92b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:33:28.861 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:33:28 smithi132 ceph-mds[69399]: -- 172.21.15.132:0/3082937261 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560cd4dbba40 con 0x560cd4d8ec00 2024-04-23T18:33:28.861 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:33:28 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-j[69395]: starting mds.j at 2024-04-23T18:33:28.861 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:33:28 smithi132 bash[69380]: 59874b8d89d27ac8fa3af79c0b45ea12c54619e5e595dffd1a4457be4c5b92b2 2024-04-23T18:33:28.862 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:33:28 smithi132 systemd[1]: Started Ceph mds.j for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:33:28.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.929+0000 7f2f53a43640 1 -- 172.21.15.132:0/2448873714 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f4c02d5c0 con 0x7f2f4c1011e0 2024-04-23T18:33:28.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.929+0000 7f2f427fc640 1 -- 172.21.15.132:0/2448873714 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f2f3c009de0 con 0x7f2f4c1011e0 2024-04-23T18:33:28.921 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:28.921 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T18:33:28.922 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 3 2024-04-23T18:33:28.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.931+0000 7f2f53a43640 1 -- 172.21.15.132:0/2448873714 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2f380a3bb0 msgr2=0x7f2f380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:28.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.931+0000 7f2f53a43640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2f380a3bb0 0x7f2f380a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2f48031500 tx=0x7f2f48005fd0 comp rx=0 tx=0).stop 2024-04-23T18:33:28.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.931+0000 7f2f53a43640 1 -- 172.21.15.132:0/2448873714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c1011e0 msgr2=0x7f2f4c19a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:28.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.931+0000 7f2f53a43640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c1011e0 0x7f2f4c19a930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00d950 tx=0x7f2f3c00de10 comp rx=0 tx=0).stop 2024-04-23T18:33:28.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.931+0000 7f2f53a43640 1 -- 172.21.15.132:0/2448873714 shutdown_connections 2024-04-23T18:33:28.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.931+0000 7f2f53a43640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2f380a3bb0 0x7f2f380a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:28.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.931+0000 7f2f53a43640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f4c101b80 0x7f2f4c1a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:28.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.931+0000 7f2f53a43640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c1011e0 0x7f2f4c19a930 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:28.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.931+0000 7f2f53a43640 1 --2- 172.21.15.132:0/2448873714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f4c100880 0x7f2f4c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:28.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.931+0000 7f2f53a43640 1 -- 172.21.15.132:0/2448873714 >> 172.21.15.132:0/2448873714 conn(0x7f2f4c0fc400 msgr2=0x7f2f4c0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:28.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.931+0000 7f2f53a43640 1 -- 172.21.15.132:0/2448873714 shutdown_connections 2024-04-23T18:33:28.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:28.931+0000 7f2f53a43640 1 -- 172.21.15.132:0/2448873714 wait complete. 2024-04-23T18:33:29.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:29 smithi148 ceph-mon[32311]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:29.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:29 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:29.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:29 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:29.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:29 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:29.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:29 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:29.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:29 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:29.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:29 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:29.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:29 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2448873714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:29.320 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:29 smithi132 ceph-mon[29178]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:29.320 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:29 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:29.320 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:29 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:29.320 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:29 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:29.320 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:29 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:29.320 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:29 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:29.320 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:29 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:29.321 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:29 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2448873714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:29.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:29 smithi157 ceph-mon[32580]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:29.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:29 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:29.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:29 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:29.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:29 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:29.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:29 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:29.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:29 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:29.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:29 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:29.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:29 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2448873714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:30.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:30 smithi132 ceph-mon[29178]: Deploying daemon mds.d on smithi132 2024-04-23T18:33:30.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:30 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.132:6834/152253190,v1:172.21.15.132:6835/152253190] up:boot 2024-04-23T18:33:30.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:30 smithi132 ceph-mon[29178]: fsmap 3 up:standby 2024-04-23T18:33:30.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:30 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:33:30.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:30 smithi148 ceph-mon[32311]: Deploying daemon mds.d on smithi132 2024-04-23T18:33:30.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:30 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.132:6834/152253190,v1:172.21.15.132:6835/152253190] up:boot 2024-04-23T18:33:30.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:30 smithi148 ceph-mon[32311]: fsmap 3 up:standby 2024-04-23T18:33:30.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:30 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:33:30.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:30 smithi157 ceph-mon[32580]: Deploying daemon mds.d on smithi132 2024-04-23T18:33:30.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:30 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.132:6834/152253190,v1:172.21.15.132:6835/152253190] up:boot 2024-04-23T18:33:30.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:30 smithi157 ceph-mon[32580]: fsmap 3 up:standby 2024-04-23T18:33:30.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:30 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:33:30.936 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:31.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.056+0000 7f876b6ef640 1 -- 172.21.15.132:0/1618980686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87641031b0 msgr2=0x7f8764103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:31.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.056+0000 7f876b6ef640 1 --2- 172.21.15.132:0/1618980686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87641031b0 0x7f8764103630 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8754009900 tx=0x7f875402a410 comp rx=0 tx=0).stop 2024-04-23T18:33:31.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.056+0000 7f876b6ef640 1 -- 172.21.15.132:0/1618980686 shutdown_connections 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.056+0000 7f876b6ef640 1 --2- 172.21.15.132:0/1618980686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8764103b70 0x7f876410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.056+0000 7f876b6ef640 1 --2- 172.21.15.132:0/1618980686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87641031b0 0x7f8764103630 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.056+0000 7f876b6ef640 1 --2- 172.21.15.132:0/1618980686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8764101fc0 0x7f87641023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.056+0000 7f876b6ef640 1 -- 172.21.15.132:0/1618980686 >> 172.21.15.132:0/1618980686 conn(0x7f87640fd550 msgr2=0x7f87640ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.056+0000 7f876b6ef640 1 -- 172.21.15.132:0/1618980686 shutdown_connections 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.056+0000 7f876b6ef640 1 -- 172.21.15.132:0/1618980686 wait complete. 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f876b6ef640 1 Processor -- start 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f876b6ef640 1 -- start start 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f876b6ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8764101fc0 0x7f87641981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f876b6ef640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87641031b0 0x7f8764198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f876b6ef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8764103b70 0x7f876419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f876b6ef640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f876410db50 con 0x7f87641031b0 2024-04-23T18:33:31.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f876b6ef640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f876410d9d0 con 0x7f8764103b70 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f876b6ef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f876410dcd0 con 0x7f8764101fc0 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f8768c8a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87641031b0 0x7f8764198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f8768c8a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87641031b0 0x7f8764198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41254/0 (socket says 172.21.15.132:41254) 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f8768c8a640 1 -- 172.21.15.132:0/2398654628 learned_addr learned my addr 172.21.15.132:0/2398654628 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f8769c8c640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8764103b70 0x7f876419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f876948b640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8764101fc0 0x7f87641981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f8768c8a640 1 -- 172.21.15.132:0/2398654628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8764101fc0 msgr2=0x7f87641981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f8768c8a640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8764101fc0 0x7f87641981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f8768c8a640 1 -- 172.21.15.132:0/2398654628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8764103b70 msgr2=0x7f876419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f8768c8a640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8764103b70 0x7f876419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.057+0000 7f8768c8a640 1 -- 172.21.15.132:0/2398654628 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876419fe00 con 0x7f87641031b0 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.058+0000 7f8768c8a640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87641031b0 0x7f8764198720 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8754031500 tx=0x7f8754003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.058+0000 7f875a7fc640 1 -- 172.21.15.132:0/2398654628 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875403f070 con 0x7f87641031b0 2024-04-23T18:33:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.058+0000 7f876b6ef640 1 -- 172.21.15.132:0/2398654628 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87641a0090 con 0x7f87641031b0 2024-04-23T18:33:31.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.058+0000 7f875a7fc640 1 -- 172.21.15.132:0/2398654628 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f875403b460 con 0x7f87641031b0 2024-04-23T18:33:31.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.058+0000 7f875a7fc640 1 -- 172.21.15.132:0/2398654628 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754035030 con 0x7f87641031b0 2024-04-23T18:33:31.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.058+0000 7f876b6ef640 1 -- 172.21.15.132:0/2398654628 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87641a0580 con 0x7f87641031b0 2024-04-23T18:33:31.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.059+0000 7f876b6ef640 1 -- 172.21.15.132:0/2398654628 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8764109360 con 0x7f87641031b0 2024-04-23T18:33:31.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.060+0000 7f875a7fc640 1 -- 172.21.15.132:0/2398654628 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8754042070 con 0x7f87641031b0 2024-04-23T18:33:31.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.061+0000 7f875a7fc640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f87500a3bb0 0x7f87500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:31.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.061+0000 7f875a7fc640 1 -- 172.21.15.132:0/2398654628 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f87540f0d70 con 0x7f87641031b0 2024-04-23T18:33:31.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.061+0000 7f876948b640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f87500a3bb0 0x7f87500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:31.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.061+0000 7f876948b640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f87500a3bb0 0x7f87500a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8760005db0 tx=0x7f8760005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:31.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.068+0000 7f875a7fc640 1 -- 172.21.15.132:0/2398654628 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87540bdbf0 con 0x7f87641031b0 2024-04-23T18:33:31.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.291+0000 7f876b6ef640 1 -- 172.21.15.132:0/2398654628 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f876402d5c0 con 0x7f87641031b0 2024-04-23T18:33:31.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.292+0000 7f875a7fc640 1 -- 172.21.15.132:0/2398654628 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f87540bd4f0 con 0x7f87641031b0 2024-04-23T18:33:31.284 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:31.284 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T18:33:31.285 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 4 2024-04-23T18:33:31.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.294+0000 7f876b6ef640 1 -- 172.21.15.132:0/2398654628 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f87500a3bb0 msgr2=0x7f87500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:31.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.294+0000 7f876b6ef640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f87500a3bb0 0x7f87500a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8760005db0 tx=0x7f8760005d40 comp rx=0 tx=0).stop 2024-04-23T18:33:31.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.294+0000 7f876b6ef640 1 -- 172.21.15.132:0/2398654628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87641031b0 msgr2=0x7f8764198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:31.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.294+0000 7f876b6ef640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87641031b0 0x7f8764198720 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8754031500 tx=0x7f8754003d50 comp rx=0 tx=0).stop 2024-04-23T18:33:31.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.294+0000 7f876b6ef640 1 -- 172.21.15.132:0/2398654628 shutdown_connections 2024-04-23T18:33:31.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.294+0000 7f876b6ef640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f87500a3bb0 0x7f87500a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:31.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.294+0000 7f876b6ef640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8764103b70 0x7f876419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:31.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.294+0000 7f876b6ef640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87641031b0 0x7f8764198720 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:31.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.294+0000 7f876b6ef640 1 --2- 172.21.15.132:0/2398654628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8764101fc0 0x7f87641981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:31.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.294+0000 7f876b6ef640 1 -- 172.21.15.132:0/2398654628 >> 172.21.15.132:0/2398654628 conn(0x7f87640fd550 msgr2=0x7f8764106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:31.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.294+0000 7f876b6ef640 1 -- 172.21.15.132:0/2398654628 shutdown_connections 2024-04-23T18:33:31.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:31.294+0000 7f876b6ef640 1 -- 172.21.15.132:0/2398654628 wait complete. 2024-04-23T18:33:31.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:31 smithi148 ceph-mon[32311]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:31.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:31 smithi157 ceph-mon[32580]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:31.556 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:31 smithi132 ceph-mon[29178]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:31.861 INFO:journalctl@ceph.mds.d.smithi132.stdout:Apr 23 18:33:31 smithi132 systemd[1]: Started Ceph mds.d for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:33:32.481 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:32 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2398654628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:32.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:32.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:32.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:32.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:32.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:32.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:32 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:32.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:32 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2398654628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:32.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:32.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:32.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:32.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:32.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:32.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:32 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:32.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:32 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2398654628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:32.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:32.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:32.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:32.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:32.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:32.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:32 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:33.299 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:33.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/2793603845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bc0107c80 msgr2=0x7f4bc010a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:33.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 --2- 172.21.15.132:0/2793603845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bc0107c80 0x7f4bc010a0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009ca0 tx=0x7f4bb802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:33:33.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/2793603845 shutdown_connections 2024-04-23T18:33:33.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 --2- 172.21.15.132:0/2793603845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc0105350 0x7f4bc0107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:33.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 --2- 172.21.15.132:0/2793603845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bc0107c80 0x7f4bc010a0d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:33.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 --2- 172.21.15.132:0/2793603845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc0102a20 0x7f4bc0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:33.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/2793603845 >> 172.21.15.132:0/2793603845 conn(0x7f4bc00fc430 msgr2=0x7f4bc00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:33.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/2793603845 shutdown_connections 2024-04-23T18:33:33.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/2793603845 wait complete. 2024-04-23T18:33:33.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 Processor -- start 2024-04-23T18:33:33.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 -- start start 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bc0102a20 0x7f4bc01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.420+0000 7f4bc7dfe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc0105350 0x7f4bc01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc7dfe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc0107c80 0x7f4bc019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc7dfe640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc010db30 con 0x7f4bc0102a20 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc7dfe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc010d9b0 con 0x7f4bc0105350 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc7dfe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc010dcb0 con 0x7f4bc0107c80 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc5b9a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bc0102a20 0x7f4bc01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc5b9a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bc0102a20 0x7f4bc01981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41290/0 (socket says 172.21.15.132:41290) 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc5b9a640 1 -- 172.21.15.132:0/160739421 learned_addr learned my addr 172.21.15.132:0/160739421 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc5399640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc0105350 0x7f4bc01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc639b640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc0107c80 0x7f4bc019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc5b9a640 1 -- 172.21.15.132:0/160739421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc0107c80 msgr2=0x7f4bc019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc5b9a640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc0107c80 0x7f4bc019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:33.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc5b9a640 1 -- 172.21.15.132:0/160739421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc0105350 msgr2=0x7f4bc01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:33.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc5b9a640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc0105350 0x7f4bc01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:33.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc5b9a640 1 -- 172.21.15.132:0/160739421 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc019fdd0 con 0x7f4bc0102a20 2024-04-23T18:33:33.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc639b640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc0107c80 0x7f4bc019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:33.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc5b9a640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bc0102a20 0x7f4bc01981b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c890 tx=0x7f4bbc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:33.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bb6ffd640 1 -- 172.21.15.132:0/160739421 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc004220 con 0x7f4bc0102a20 2024-04-23T18:33:33.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bb6ffd640 1 -- 172.21.15.132:0/160739421 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bbc0043c0 con 0x7f4bc0102a20 2024-04-23T18:33:33.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/160739421 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc01a00c0 con 0x7f4bc0102a20 2024-04-23T18:33:33.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bb6ffd640 1 -- 172.21.15.132:0/160739421 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc004a00 con 0x7f4bc0102a20 2024-04-23T18:33:33.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.421+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/160739421 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc01a0660 con 0x7f4bc0102a20 2024-04-23T18:33:33.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.422+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/160739421 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc00fdf20 con 0x7f4bc0102a20 2024-04-23T18:33:33.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.423+0000 7f4bb6ffd640 1 -- 172.21.15.132:0/160739421 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4bbc0079b0 con 0x7f4bc0102a20 2024-04-23T18:33:33.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.424+0000 7f4bb6ffd640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4bac0a3c30 0x7f4bac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:33.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.424+0000 7f4bc5399640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4bac0a3c30 0x7f4bac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:33.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.424+0000 7f4bb6ffd640 1 -- 172.21.15.132:0/160739421 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4bbc0cfc20 con 0x7f4bc0102a20 2024-04-23T18:33:33.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.425+0000 7f4bc5399640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4bac0a3c30 0x7f4bac0a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4bc01996c0 tx=0x7f4bb000a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:33.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.429+0000 7f4bb6ffd640 1 -- 172.21.15.132:0/160739421 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bbc014030 con 0x7f4bc0102a20 2024-04-23T18:33:33.441 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:33 smithi157 ceph-mon[32580]: Deploying daemon mds.e on smithi148 2024-04-23T18:33:33.441 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:33 smithi157 ceph-mon[32580]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:33.442 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:33 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.132:6836/1545569950,v1:172.21.15.132:6837/1545569950] up:boot 2024-04-23T18:33:33.442 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:33 smithi157 ceph-mon[32580]: fsmap 4 up:standby 2024-04-23T18:33:33.442 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:33 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:33:33.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:33 smithi148 ceph-mon[32311]: Deploying daemon mds.e on smithi148 2024-04-23T18:33:33.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:33 smithi148 ceph-mon[32311]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:33.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:33 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.132:6836/1545569950,v1:172.21.15.132:6837/1545569950] up:boot 2024-04-23T18:33:33.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:33 smithi148 ceph-mon[32311]: fsmap 4 up:standby 2024-04-23T18:33:33.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:33 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:33:33.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:33 smithi132 ceph-mon[29178]: Deploying daemon mds.e on smithi148 2024-04-23T18:33:33.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:33 smithi132 ceph-mon[29178]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:33.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:33 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.132:6836/1545569950,v1:172.21.15.132:6837/1545569950] up:boot 2024-04-23T18:33:33.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:33 smithi132 ceph-mon[29178]: fsmap 4 up:standby 2024-04-23T18:33:33.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:33 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:33:33.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.644+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/160739421 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bc002d5c0 con 0x7f4bc0102a20 2024-04-23T18:33:33.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.644+0000 7f4bb6ffd640 1 -- 172.21.15.132:0/160739421 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3877 (secure 0 0 0) 0x7f4bbc09caa0 con 0x7f4bc0102a20 2024-04-23T18:33:33.637 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:33.637 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T18:33:33.637 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 5 2024-04-23T18:33:33.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.646+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/160739421 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4bac0a3c30 msgr2=0x7f4bac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:33.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.646+0000 7f4bc7dfe640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4bac0a3c30 0x7f4bac0a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4bc01996c0 tx=0x7f4bb000a250 comp rx=0 tx=0).stop 2024-04-23T18:33:33.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.646+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/160739421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bc0102a20 msgr2=0x7f4bc01981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:33.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.646+0000 7f4bc7dfe640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bc0102a20 0x7f4bc01981b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c890 tx=0x7f4bbc00cd50 comp rx=0 tx=0).stop 2024-04-23T18:33:33.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.646+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/160739421 shutdown_connections 2024-04-23T18:33:33.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.646+0000 7f4bc7dfe640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4bac0a3c30 0x7f4bac0a60f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:33.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.646+0000 7f4bc7dfe640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc0107c80 0x7f4bc019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:33.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.646+0000 7f4bc7dfe640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc0105350 0x7f4bc01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:33.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.646+0000 7f4bc7dfe640 1 --2- 172.21.15.132:0/160739421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bc0102a20 0x7f4bc01981b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:33.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.646+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/160739421 >> 172.21.15.132:0/160739421 conn(0x7f4bc00fc430 msgr2=0x7f4bc0103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:33.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.646+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/160739421 shutdown_connections 2024-04-23T18:33:33.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:33.646+0000 7f4bc7dfe640 1 -- 172.21.15.132:0/160739421 wait complete. 2024-04-23T18:33:34.195 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:33:34 smithi148 systemd[1]: Starting Ceph mds.e for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:33:34.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:34 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/160739421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:34.514 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:33:34 smithi148 podman[61891]: 2024-04-23 18:33:34.210324264 +0000 UTC m=+0.019292792 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:33:34.515 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:33:34 smithi148 podman[61891]: 2024-04-23 18:33:34.308390921 +0000 UTC m=+0.117359434 container create 2c46f700e04ffea43a0eb21adc45d03dbf5636003d4db9492cf1310baf7b4055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-23T18:33:34.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:34 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/160739421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:34.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:34 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/160739421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:35.014 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:33:34 smithi148 podman[61891]: 2024-04-23 18:33:34.550327245 +0000 UTC m=+0.359295758 container init 2c46f700e04ffea43a0eb21adc45d03dbf5636003d4db9492cf1310baf7b4055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-23T18:33:35.015 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:33:34 smithi148 podman[61891]: 2024-04-23 18:33:34.552796104 +0000 UTC m=+0.361764620 container start 2c46f700e04ffea43a0eb21adc45d03dbf5636003d4db9492cf1310baf7b4055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-23T18:33:35.015 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:33:34 smithi148 ceph-mds[61910]: -- 172.21.15.148:0/4074953046 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e13187a40 con 0x561e1315a800 2024-04-23T18:33:35.015 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:33:34 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-e[61906]: starting mds.e at 2024-04-23T18:33:35.015 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:33:34 smithi148 bash[61891]: 2c46f700e04ffea43a0eb21adc45d03dbf5636003d4db9492cf1310baf7b4055 2024-04-23T18:33:35.015 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:33:34 smithi148 systemd[1]: Started Ceph mds.e for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:33:35.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:35 smithi148 ceph-mon[32311]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:35.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:35.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:35.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:35.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:35.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:35.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:35 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:35.524 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:35 smithi157 ceph-mon[32580]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:35.524 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:35.524 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:35.524 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:35.524 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:35.524 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:35.524 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:35 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:35.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:35 smithi132 ceph-mon[29178]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:35.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:35.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:35 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:35.650 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:35.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.768+0000 7f1bf5a43640 1 -- 172.21.15.132:0/2032666168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf01031b0 msgr2=0x7f1bf0103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.768+0000 7f1bf5a43640 1 --2- 172.21.15.132:0/2032666168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf01031b0 0x7f1bf0103630 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009900 tx=0x7f1be402a410 comp rx=0 tx=0).stop 2024-04-23T18:33:35.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.768+0000 7f1bf5a43640 1 -- 172.21.15.132:0/2032666168 shutdown_connections 2024-04-23T18:33:35.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.768+0000 7f1bf5a43640 1 --2- 172.21.15.132:0/2032666168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf0103b70 0x7f1bf010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.768+0000 7f1bf5a43640 1 --2- 172.21.15.132:0/2032666168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf01031b0 0x7f1bf0103630 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.768+0000 7f1bf5a43640 1 --2- 172.21.15.132:0/2032666168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf0101fc0 0x7f1bf01023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.768+0000 7f1bf5a43640 1 -- 172.21.15.132:0/2032666168 >> 172.21.15.132:0/2032666168 conn(0x7f1bf00fd550 msgr2=0x7f1bf00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:35.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.768+0000 7f1bf5a43640 1 -- 172.21.15.132:0/2032666168 shutdown_connections 2024-04-23T18:33:35.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.768+0000 7f1bf5a43640 1 -- 172.21.15.132:0/2032666168 wait complete. 2024-04-23T18:33:35.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1bf5a43640 1 Processor -- start 2024-04-23T18:33:35.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1bf5a43640 1 -- start start 2024-04-23T18:33:35.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1bf5a43640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf0101fc0 0x7f1bf019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1bf5a43640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf01031b0 0x7f1bf019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1bf5a43640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf0103b70 0x7f1bf01a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1bf5a43640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf00751d0 con 0x7f1bf0101fc0 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1bf5a43640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf0075050 con 0x7f1bf0103b70 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1bf5a43640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf0075350 con 0x7f1bf01031b0 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1beeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf0101fc0 0x7f1bf019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1beeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf0101fc0 0x7f1bf019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41296/0 (socket says 172.21.15.132:41296) 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1beeffd640 1 -- 172.21.15.132:0/380468786 learned_addr learned my addr 172.21.15.132:0/380468786 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1bee7fc640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf01031b0 0x7f1bf019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1bef7fe640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf0103b70 0x7f1bf01a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1beeffd640 1 -- 172.21.15.132:0/380468786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf01031b0 msgr2=0x7f1bf019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1beeffd640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf01031b0 0x7f1bf019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1beeffd640 1 -- 172.21.15.132:0/380468786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf0103b70 msgr2=0x7f1bf01a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1beeffd640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf0103b70 0x7f1bf01a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.769+0000 7f1beeffd640 1 -- 172.21.15.132:0/380468786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf01a2020 con 0x7f1bf0101fc0 2024-04-23T18:33:35.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.770+0000 7f1beeffd640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf0101fc0 0x7f1bf019a400 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1be000d770 tx=0x7f1be000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:35.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.770+0000 7f1bf4a41640 1 -- 172.21.15.132:0/380468786 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0017070 con 0x7f1bf0101fc0 2024-04-23T18:33:35.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.770+0000 7f1bf4a41640 1 -- 172.21.15.132:0/380468786 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1be0009c30 con 0x7f1bf0101fc0 2024-04-23T18:33:35.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.770+0000 7f1bf5a43640 1 -- 172.21.15.132:0/380468786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf01a2310 con 0x7f1bf0101fc0 2024-04-23T18:33:35.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.770+0000 7f1bf4a41640 1 -- 172.21.15.132:0/380468786 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0003cc0 con 0x7f1bf0101fc0 2024-04-23T18:33:35.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.770+0000 7f1bf5a43640 1 -- 172.21.15.132:0/380468786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf01a2830 con 0x7f1bf0101fc0 2024-04-23T18:33:35.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.771+0000 7f1bf5a43640 1 -- 172.21.15.132:0/380468786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf0108b00 con 0x7f1bf0101fc0 2024-04-23T18:33:35.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.772+0000 7f1bf4a41640 1 -- 172.21.15.132:0/380468786 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1be0003e60 con 0x7f1bf0101fc0 2024-04-23T18:33:35.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.772+0000 7f1bf4a41640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1bd80a3bb0 0x7f1bd80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.773+0000 7f1bee7fc640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1bd80a3bb0 0x7f1bd80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.773+0000 7f1bf4a41640 1 -- 172.21.15.132:0/380468786 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1be00cf8e0 con 0x7f1bf0101fc0 2024-04-23T18:33:35.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.773+0000 7f1bee7fc640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1bd80a3bb0 0x7f1bd80a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1be4000c00 tx=0x7f1be40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:35.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.777+0000 7f1bf4a41640 1 -- 172.21.15.132:0/380468786 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1be0009de0 con 0x7f1bf0101fc0 2024-04-23T18:33:35.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.990+0000 7f1bf5a43640 1 -- 172.21.15.132:0/380468786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bf002d5c0 con 0x7f1bf0101fc0 2024-04-23T18:33:35.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.990+0000 7f1bf4a41640 1 -- 172.21.15.132:0/380468786 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7f1be009c070 con 0x7f1bf0101fc0 2024-04-23T18:33:35.983 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:35.983 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T18:33:35.983 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 6 2024-04-23T18:33:35.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.993+0000 7f1bf5a43640 1 -- 172.21.15.132:0/380468786 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1bd80a3bb0 msgr2=0x7f1bd80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.993+0000 7f1bf5a43640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1bd80a3bb0 0x7f1bd80a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1be4000c00 tx=0x7f1be40314a0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.993+0000 7f1bf5a43640 1 -- 172.21.15.132:0/380468786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf0101fc0 msgr2=0x7f1bf019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.993+0000 7f1bf5a43640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf0101fc0 0x7f1bf019a400 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1be000d770 tx=0x7f1be000dc30 comp rx=0 tx=0).stop 2024-04-23T18:33:35.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.993+0000 7f1bf5a43640 1 -- 172.21.15.132:0/380468786 shutdown_connections 2024-04-23T18:33:35.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.993+0000 7f1bf5a43640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1bd80a3bb0 0x7f1bd80a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.993+0000 7f1bf5a43640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bf01031b0 0x7f1bf019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.993+0000 7f1bf5a43640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf0103b70 0x7f1bf01a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.993+0000 7f1bf5a43640 1 --2- 172.21.15.132:0/380468786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bf0101fc0 0x7f1bf019a400 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.993+0000 7f1bf5a43640 1 -- 172.21.15.132:0/380468786 >> 172.21.15.132:0/380468786 conn(0x7f1bf00fd550 msgr2=0x7f1bf00fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:35.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.993+0000 7f1bf5a43640 1 -- 172.21.15.132:0/380468786 shutdown_connections 2024-04-23T18:33:35.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:35.993+0000 7f1bf5a43640 1 -- 172.21.15.132:0/380468786 wait complete. 2024-04-23T18:33:36.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:36 smithi132 ceph-mon[29178]: Deploying daemon mds.f on smithi157 2024-04-23T18:33:36.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:36 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.148:6834/397710093,v1:172.21.15.148:6835/397710093] up:boot 2024-04-23T18:33:36.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:36 smithi132 ceph-mon[29178]: fsmap 5 up:standby 2024-04-23T18:33:36.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:36 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:33:36.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:36 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/380468786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:36.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:36 smithi157 ceph-mon[32580]: Deploying daemon mds.f on smithi157 2024-04-23T18:33:36.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:36 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.148:6834/397710093,v1:172.21.15.148:6835/397710093] up:boot 2024-04-23T18:33:36.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:36 smithi157 ceph-mon[32580]: fsmap 5 up:standby 2024-04-23T18:33:36.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:36 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:33:36.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:36 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/380468786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:36.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:36 smithi148 ceph-mon[32311]: Deploying daemon mds.f on smithi157 2024-04-23T18:33:36.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:36 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.148:6834/397710093,v1:172.21.15.148:6835/397710093] up:boot 2024-04-23T18:33:36.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:36 smithi148 ceph-mon[32311]: fsmap 5 up:standby 2024-04-23T18:33:36.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:36 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:33:36.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:36 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/380468786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:37.208 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:33:37 smithi157 podman[62864]: 2024-04-23 18:33:37.147508883 +0000 UTC m=+0.138413163 container create fdfc1d81fa7fa7291df9f8c7526ad029fbbc9116728a1ce3792e56e3add5a545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-23T18:33:37.467 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:37 smithi157 ceph-mon[32580]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:37.467 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:33:37 smithi157 podman[62864]: 2024-04-23 18:33:37.383814701 +0000 UTC m=+0.374718983 container init fdfc1d81fa7fa7291df9f8c7526ad029fbbc9116728a1ce3792e56e3add5a545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-23T18:33:37.467 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:33:37 smithi157 podman[62864]: 2024-04-23 18:33:37.386467865 +0000 UTC m=+0.377372152 container start fdfc1d81fa7fa7291df9f8c7526ad029fbbc9116728a1ce3792e56e3add5a545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:33:37.467 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:33:37 smithi157 ceph-mds[62883]: -- 172.21.15.157:0/2635468831 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f7fba05a40 con 0x55f7fb9d9000 2024-04-23T18:33:37.467 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:33:37 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f[62879]: starting mds.f at 2024-04-23T18:33:37.467 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:33:37 smithi157 bash[62864]: fdfc1d81fa7fa7291df9f8c7526ad029fbbc9116728a1ce3792e56e3add5a545 2024-04-23T18:33:37.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:37 smithi148 ceph-mon[32311]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:37.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:37 smithi132 ceph-mon[29178]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:37.789 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:33:37 smithi157 systemd[1]: Started Ceph mds.f for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:33:37.997 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:38.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.115+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/2338543468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec4100270 msgr2=0x7f7ec4100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:38.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.115+0000 7f7ecc3bf640 1 --2- 172.21.15.132:0/2338543468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec4100270 0x7f7ec4100690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009930 tx=0x7f7ec002a420 comp rx=0 tx=0).stop 2024-04-23T18:33:38.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.115+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/2338543468 shutdown_connections 2024-04-23T18:33:38.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.115+0000 7f7ecc3bf640 1 --2- 172.21.15.132:0/2338543468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ec4103240 0x7f7ec41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.115+0000 7f7ecc3bf640 1 --2- 172.21.15.132:0/2338543468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ec4103c00 0x7f7ec410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.115+0000 7f7ecc3bf640 1 --2- 172.21.15.132:0/2338543468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec4100270 0x7f7ec4100690 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.115+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/2338543468 >> 172.21.15.132:0/2338543468 conn(0x7f7ec40fbbc0 msgr2=0x7f7ec40fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.115+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/2338543468 shutdown_connections 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.115+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/2338543468 wait complete. 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7ecc3bf640 1 Processor -- start 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7ecc3bf640 1 -- start start 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7ecc3bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ec4100270 0x7f7ec4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7ecc3bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ec4103240 0x7f7ec41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7ecc3bf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec4103c00 0x7f7ec419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7ecc3bf640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec410dad0 con 0x7f7ec4103c00 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7ecc3bf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec410d950 con 0x7f7ec4100270 2024-04-23T18:33:38.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7ecc3bf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec410dc50 con 0x7f7ec4103240 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7eca95c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec4103c00 0x7f7ec419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7eca95c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec4103c00 0x7f7ec419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41316/0 (socket says 172.21.15.132:41316) 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7eca95c640 1 -- 172.21.15.132:0/3399056637 learned_addr learned my addr 172.21.15.132:0/3399056637 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7ec995a640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ec4103240 0x7f7ec41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7eca15b640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ec4100270 0x7f7ec4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7eca95c640 1 -- 172.21.15.132:0/3399056637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ec4103240 msgr2=0x7f7ec41986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7eca95c640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ec4103240 0x7f7ec41986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7eca95c640 1 -- 172.21.15.132:0/3399056637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ec4100270 msgr2=0x7f7ec4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7eca95c640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ec4100270 0x7f7ec4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7eca95c640 1 -- 172.21.15.132:0/3399056637 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec419fd90 con 0x7f7ec4103c00 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7ec995a640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ec4103240 0x7f7ec41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.116+0000 7f7eca95c640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec4103c00 0x7f7ec419f720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00cce0 tx=0x7f7ebc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:38.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.117+0000 7f7ebb7fe640 1 -- 172.21.15.132:0/3399056637 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00f450 con 0x7f7ec4103c00 2024-04-23T18:33:38.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.117+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/3399056637 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec41a0080 con 0x7f7ec4103c00 2024-04-23T18:33:38.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.117+0000 7f7ebb7fe640 1 -- 172.21.15.132:0/3399056637 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ebc00faa0 con 0x7f7ec4103c00 2024-04-23T18:33:38.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.117+0000 7f7ebb7fe640 1 -- 172.21.15.132:0/3399056637 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc018b70 con 0x7f7ec4103c00 2024-04-23T18:33:38.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.117+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/3399056637 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec41a0580 con 0x7f7ec4103c00 2024-04-23T18:33:38.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.118+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/3399056637 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec41093f0 con 0x7f7ec4103c00 2024-04-23T18:33:38.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.119+0000 7f7ebb7fe640 1 -- 172.21.15.132:0/3399056637 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7ebc018d10 con 0x7f7ec4103c00 2024-04-23T18:33:38.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.120+0000 7f7ebb7fe640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7eb00a3bb0 0x7f7eb00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.120+0000 7f7eca15b640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7eb00a3bb0 0x7f7eb00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.120+0000 7f7ebb7fe640 1 -- 172.21.15.132:0/3399056637 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7ebc0d0450 con 0x7f7ec4103c00 2024-04-23T18:33:38.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.120+0000 7f7eca15b640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7eb00a3bb0 0x7f7eb00a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0006010 tx=0x7f7ec00095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:38.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.124+0000 7f7ebb7fe640 1 -- 172.21.15.132:0/3399056637 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ebc014030 con 0x7f7ec4103c00 2024-04-23T18:33:38.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.343+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/3399056637 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ec4062b50 con 0x7f7ec4103c00 2024-04-23T18:33:38.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.343+0000 7f7ebb7fe640 1 -- 172.21.15.132:0/3399056637 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f7ebc09d2d0 con 0x7f7ec4103c00 2024-04-23T18:33:38.336 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:38.336 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T18:33:38.336 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 7 2024-04-23T18:33:38.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.345+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/3399056637 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7eb00a3bb0 msgr2=0x7f7eb00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:38.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.345+0000 7f7ecc3bf640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7eb00a3bb0 0x7f7eb00a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0006010 tx=0x7f7ec00095f0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.345+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/3399056637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec4103c00 msgr2=0x7f7ec419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:38.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.345+0000 7f7ecc3bf640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec4103c00 0x7f7ec419f720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00cce0 tx=0x7f7ebc00a620 comp rx=0 tx=0).stop 2024-04-23T18:33:38.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.345+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/3399056637 shutdown_connections 2024-04-23T18:33:38.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.345+0000 7f7ecc3bf640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7eb00a3bb0 0x7f7eb00a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.345+0000 7f7ecc3bf640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ec4103240 0x7f7ec41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.345+0000 7f7ecc3bf640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec4103c00 0x7f7ec419f720 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.345+0000 7f7ecc3bf640 1 --2- 172.21.15.132:0/3399056637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ec4100270 0x7f7ec4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.345+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/3399056637 >> 172.21.15.132:0/3399056637 conn(0x7f7ec40fbbc0 msgr2=0x7f7ec41065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:38.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.345+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/3399056637 shutdown_connections 2024-04-23T18:33:38.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:38.345+0000 7f7ecc3bf640 1 -- 172.21.15.132:0/3399056637 wait complete. 2024-04-23T18:33:38.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:38.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:38.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:38.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:38.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:38.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:38.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:38 smithi132 ceph-mon[29178]: Deploying daemon mds.c on smithi157 2024-04-23T18:33:38.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:38 smithi132 ceph-mon[29178]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:38.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:38 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.157:6834/4121037781,v1:172.21.15.157:6835/4121037781] up:boot 2024-04-23T18:33:38.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:38 smithi132 ceph-mon[29178]: fsmap 6 up:standby 2024-04-23T18:33:38.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:38 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:33:38.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:38 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3399056637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:38.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:38.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:38.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:38.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:38.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:38.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:38.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:38 smithi157 ceph-mon[32580]: Deploying daemon mds.c on smithi157 2024-04-23T18:33:38.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:38 smithi157 ceph-mon[32580]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:38.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:38 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.157:6834/4121037781,v1:172.21.15.157:6835/4121037781] up:boot 2024-04-23T18:33:38.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:38 smithi157 ceph-mon[32580]: fsmap 6 up:standby 2024-04-23T18:33:38.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:38 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:33:38.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:38 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3399056637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:39.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:39.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:39.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:39.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:39.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:39.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:39.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:38 smithi148 ceph-mon[32311]: Deploying daemon mds.c on smithi157 2024-04-23T18:33:39.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:38 smithi148 ceph-mon[32311]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:39.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:38 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.157:6834/4121037781,v1:172.21.15.157:6835/4121037781] up:boot 2024-04-23T18:33:39.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:38 smithi148 ceph-mon[32311]: fsmap 6 up:standby 2024-04-23T18:33:39.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:38 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:33:39.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:38 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3399056637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:39.790 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:33:39 smithi157 systemd[1]: Starting Ceph mds.c for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:33:40.189 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:33:39 smithi157 podman[63176]: 2024-04-23 18:33:39.829065882 +0000 UTC m=+0.017991224 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:33:40.189 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:33:39 smithi157 podman[63176]: 2024-04-23 18:33:39.936540648 +0000 UTC m=+0.125465976 container create e805f720b6eed34274a2f74f5d4a695d384a87a072e39c21bdfe302bc010a913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-23T18:33:40.190 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:33:40 smithi157 podman[63176]: 2024-04-23 18:33:40.197737179 +0000 UTC m=+0.386662509 container init e805f720b6eed34274a2f74f5d4a695d384a87a072e39c21bdfe302bc010a913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-23T18:33:40.350 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:40.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.469+0000 7f43c327b640 1 -- 172.21.15.132:0/3479929008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc102040 msgr2=0x7f43bc1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.469+0000 7f43c327b640 1 --2- 172.21.15.132:0/3479929008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc102040 0x7f43bc1024c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009900 tx=0x7f43b002a410 comp rx=0 tx=0).stop 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.469+0000 7f43c327b640 1 -- 172.21.15.132:0/3479929008 shutdown_connections 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.469+0000 7f43c327b640 1 --2- 172.21.15.132:0/3479929008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43bc102a00 0x7f43bc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.469+0000 7f43c327b640 1 --2- 172.21.15.132:0/3479929008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc102040 0x7f43bc1024c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.469+0000 7f43c327b640 1 --2- 172.21.15.132:0/3479929008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc100e60 0x7f43bc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.469+0000 7f43c327b640 1 -- 172.21.15.132:0/3479929008 >> 172.21.15.132:0/3479929008 conn(0x7f43bc0fc440 msgr2=0x7f43bc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.469+0000 7f43c327b640 1 -- 172.21.15.132:0/3479929008 shutdown_connections 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.469+0000 7f43c327b640 1 -- 172.21.15.132:0/3479929008 wait complete. 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c327b640 1 Processor -- start 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c327b640 1 -- start start 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c327b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc100e60 0x7f43bc1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c327b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43bc102040 0x7f43bc197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c327b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc102a00 0x7f43bc19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c327b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc10ca00 con 0x7f43bc100e60 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c327b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc10c880 con 0x7f43bc102a00 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c327b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc10cb80 con 0x7f43bc102040 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c1017640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc100e60 0x7f43bc1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c1017640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc100e60 0x7f43bc1970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55340/0 (socket says 172.21.15.132:55340) 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c1017640 1 -- 172.21.15.132:0/3709150627 learned_addr learned my addr 172.21.15.132:0/3709150627 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c1818640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc102a00 0x7f43bc19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c0816640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43bc102040 0x7f43bc197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c1017640 1 -- 172.21.15.132:0/3709150627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43bc102040 msgr2=0x7f43bc197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c1017640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43bc102040 0x7f43bc197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c1017640 1 -- 172.21.15.132:0/3709150627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc102a00 msgr2=0x7f43bc19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c1017640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc102a00 0x7f43bc19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c1017640 1 -- 172.21.15.132:0/3709150627 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43bc19ed10 con 0x7f43bc100e60 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c1818640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc102a00 0x7f43bc19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:40.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c1017640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc100e60 0x7f43bc1970f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43b400ebc0 tx=0x7f43b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43ba7fc640 1 -- 172.21.15.132:0/3709150627 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4003b70 con 0x7f43bc100e60 2024-04-23T18:33:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43ba7fc640 1 -- 172.21.15.132:0/3709150627 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b4003d10 con 0x7f43bc100e60 2024-04-23T18:33:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c327b640 1 -- 172.21.15.132:0/3709150627 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43bc19efa0 con 0x7f43bc100e60 2024-04-23T18:33:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43ba7fc640 1 -- 172.21.15.132:0/3709150627 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4019800 con 0x7f43bc100e60 2024-04-23T18:33:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.470+0000 7f43c327b640 1 -- 172.21.15.132:0/3709150627 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43bc19f460 con 0x7f43bc100e60 2024-04-23T18:33:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.471+0000 7f43c327b640 1 -- 172.21.15.132:0/3709150627 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43bc1082d0 con 0x7f43bc100e60 2024-04-23T18:33:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.472+0000 7f43ba7fc640 1 -- 172.21.15.132:0/3709150627 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f43b40199a0 con 0x7f43bc100e60 2024-04-23T18:33:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.473+0000 7f43ba7fc640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f43a40a3bb0 0x7f43a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.473+0000 7f43c0816640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f43a40a3bb0 0x7f43a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.473+0000 7f43ba7fc640 1 -- 172.21.15.132:0/3709150627 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f43b40d0920 con 0x7f43bc100e60 2024-04-23T18:33:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.474+0000 7f43c0816640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f43a40a3bb0 0x7f43a40a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f43bc198600 tx=0x7f43b00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:40.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.478+0000 7f43ba7fc640 1 -- 172.21.15.132:0/3709150627 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43b409d7a0 con 0x7f43bc100e60 2024-04-23T18:33:40.540 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:33:40 smithi157 podman[63176]: 2024-04-23 18:33:40.200890158 +0000 UTC m=+0.389815487 container start e805f720b6eed34274a2f74f5d4a695d384a87a072e39c21bdfe302bc010a913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-23T18:33:40.540 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:33:40 smithi157 ceph-mds[63196]: -- 172.21.15.157:0/1434827289 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d24f369a40 con 0x55d24f33d000 2024-04-23T18:33:40.540 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:33:40 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-c[63192]: starting mds.c at 2024-04-23T18:33:40.540 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:33:40 smithi157 bash[63176]: e805f720b6eed34274a2f74f5d4a695d384a87a072e39c21bdfe302bc010a913 2024-04-23T18:33:40.540 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:33:40 smithi157 systemd[1]: Started Ceph mds.c for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:33:40.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.694+0000 7f43c327b640 1 -- 172.21.15.132:0/3709150627 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43bc02d5c0 con 0x7f43bc100e60 2024-04-23T18:33:40.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.694+0000 7f43ba7fc640 1 -- 172.21.15.132:0/3709150627 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f43b409d0a0 con 0x7f43bc100e60 2024-04-23T18:33:40.687 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:40.687 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T18:33:40.687 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 7 2024-04-23T18:33:40.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.696+0000 7f43c327b640 1 -- 172.21.15.132:0/3709150627 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f43a40a3bb0 msgr2=0x7f43a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:40.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.696+0000 7f43c327b640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f43a40a3bb0 0x7f43a40a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f43bc198600 tx=0x7f43b00314a0 comp rx=0 tx=0).stop 2024-04-23T18:33:40.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.696+0000 7f43c327b640 1 -- 172.21.15.132:0/3709150627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc100e60 msgr2=0x7f43bc1970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:40.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.696+0000 7f43c327b640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc100e60 0x7f43bc1970f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43b400ebc0 tx=0x7f43b400c6a0 comp rx=0 tx=0).stop 2024-04-23T18:33:40.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.696+0000 7f43c327b640 1 -- 172.21.15.132:0/3709150627 shutdown_connections 2024-04-23T18:33:40.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.696+0000 7f43c327b640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f43a40a3bb0 0x7f43a40a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:40.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.696+0000 7f43c327b640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43bc102040 0x7f43bc197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:40.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.696+0000 7f43c327b640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc102a00 0x7f43bc19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:40.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.696+0000 7f43c327b640 1 --2- 172.21.15.132:0/3709150627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc100e60 0x7f43bc1970f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:40.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.696+0000 7f43c327b640 1 -- 172.21.15.132:0/3709150627 >> 172.21.15.132:0/3709150627 conn(0x7f43bc0fc440 msgr2=0x7f43bc1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:40.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.696+0000 7f43c327b640 1 -- 172.21.15.132:0/3709150627 shutdown_connections 2024-04-23T18:33:40.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:40.696+0000 7f43c327b640 1 -- 172.21.15.132:0/3709150627 wait complete. 2024-04-23T18:33:41.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:41 smithi148 ceph-mon[32311]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:41.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:41 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:41.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:41 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:41.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:41 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:41.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:41 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:41.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:41 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:41.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:41 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:41.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:41 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3709150627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:41 smithi157 ceph-mon[32580]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:41 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:41 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:41 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:41 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:41 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:41 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:41.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:41 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3709150627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:41.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:41 smithi132 ceph-mon[29178]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:41.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:41 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:41.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:41 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:41.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:41 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:41.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:41 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:41.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:41 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:41.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:41 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:41.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:41 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3709150627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:42.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:42 smithi148 ceph-mon[32311]: Deploying daemon mds.h on smithi148 2024-04-23T18:33:42.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:42 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.157:6836/2670322328,v1:172.21.15.157:6837/2670322328] up:boot 2024-04-23T18:33:42.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:42 smithi148 ceph-mon[32311]: fsmap 7 up:standby 2024-04-23T18:33:42.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:42 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:33:42.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:42 smithi157 ceph-mon[32580]: Deploying daemon mds.h on smithi148 2024-04-23T18:33:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:42 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.157:6836/2670322328,v1:172.21.15.157:6837/2670322328] up:boot 2024-04-23T18:33:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:42 smithi157 ceph-mon[32580]: fsmap 7 up:standby 2024-04-23T18:33:42.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:42 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:33:42.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:42 smithi132 ceph-mon[29178]: Deploying daemon mds.h on smithi148 2024-04-23T18:33:42.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:42 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.157:6836/2670322328,v1:172.21.15.157:6837/2670322328] up:boot 2024-04-23T18:33:42.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:42 smithi132 ceph-mon[29178]: fsmap 7 up:standby 2024-04-23T18:33:42.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:42 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:33:42.700 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:42.726 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:33:42 smithi148 systemd[1]: Starting Ceph mds.h for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:33:42.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.819+0000 7fa300907640 1 -- 172.21.15.132:0/1860850030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2fc101fa0 msgr2=0x7fa2fc1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:42.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.819+0000 7fa300907640 1 --2- 172.21.15.132:0/1860850030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2fc101fa0 0x7fa2fc1023c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009ca0 tx=0x7fa2f402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:33:42.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.819+0000 7fa300907640 1 -- 172.21.15.132:0/1860850030 shutdown_connections 2024-04-23T18:33:42.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.819+0000 7fa300907640 1 --2- 172.21.15.132:0/1860850030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2fc103190 0x7fa2fc103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.819+0000 7fa300907640 1 --2- 172.21.15.132:0/1860850030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2fc103b50 0x7fa2fc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.819+0000 7fa300907640 1 --2- 172.21.15.132:0/1860850030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2fc101fa0 0x7fa2fc1023c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.819+0000 7fa300907640 1 -- 172.21.15.132:0/1860850030 >> 172.21.15.132:0/1860850030 conn(0x7fa2fc0fd530 msgr2=0x7fa2fc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.819+0000 7fa300907640 1 -- 172.21.15.132:0/1860850030 shutdown_connections 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.819+0000 7fa300907640 1 -- 172.21.15.132:0/1860850030 wait complete. 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.820+0000 7fa300907640 1 Processor -- start 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.820+0000 7fa300907640 1 -- start start 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.820+0000 7fa300907640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2fc101fa0 0x7fa2fc1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.820+0000 7fa300907640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2fc103190 0x7fa2fc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.820+0000 7fa300907640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2fc103b50 0x7fa2fc19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.820+0000 7fa300907640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc10dae0 con 0x7fa2fc103b50 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.820+0000 7fa300907640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc10d960 con 0x7fa2fc103190 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.820+0000 7fa300907640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc10dc60 con 0x7fa2fc101fa0 2024-04-23T18:33:42.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2fad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2fc103b50 0x7fa2fc19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2fad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2fc103b50 0x7fa2fc19f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55358/0 (socket says 172.21.15.132:55358) 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2fad9d640 1 -- 172.21.15.132:0/4072300593 learned_addr learned my addr 172.21.15.132:0/4072300593 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2fa59c640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2fc101fa0 0x7fa2fc1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2f9d9b640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2fc103190 0x7fa2fc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2fad9d640 1 -- 172.21.15.132:0/4072300593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2fc101fa0 msgr2=0x7fa2fc1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2fad9d640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2fc101fa0 0x7fa2fc1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2fad9d640 1 -- 172.21.15.132:0/4072300593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2fc103190 msgr2=0x7fa2fc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2fad9d640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2fc103190 0x7fa2fc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2fad9d640 1 -- 172.21.15.132:0/4072300593 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc19fe10 con 0x7fa2fc103b50 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2f9d9b640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2fc103190 0x7fa2fc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2fad9d640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2fc103b50 0x7fa2fc19f7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00e970 tx=0x7fa2ec00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2eb7fe640 1 -- 172.21.15.132:0/4072300593 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec00ccc0 con 0x7fa2fc103b50 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2eb7fe640 1 -- 172.21.15.132:0/4072300593 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2ec00ce60 con 0x7fa2fc103b50 2024-04-23T18:33:42.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa300907640 1 -- 172.21.15.132:0/4072300593 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2fc1a0100 con 0x7fa2fc103b50 2024-04-23T18:33:42.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa2eb7fe640 1 -- 172.21.15.132:0/4072300593 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec010430 con 0x7fa2fc103b50 2024-04-23T18:33:42.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.821+0000 7fa300907640 1 -- 172.21.15.132:0/4072300593 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2fc1a0600 con 0x7fa2fc103b50 2024-04-23T18:33:42.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.822+0000 7fa300907640 1 -- 172.21.15.132:0/4072300593 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2fc1092e0 con 0x7fa2fc103b50 2024-04-23T18:33:42.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.823+0000 7fa2eb7fe640 1 -- 172.21.15.132:0/4072300593 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa2ec004ff0 con 0x7fa2fc103b50 2024-04-23T18:33:42.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.824+0000 7fa2eb7fe640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa2e40a3bb0 0x7fa2e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:42.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.824+0000 7fa2fa59c640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa2e40a3bb0 0x7fa2e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:42.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.824+0000 7fa2eb7fe640 1 -- 172.21.15.132:0/4072300593 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa2ec0d04a0 con 0x7fa2fc103b50 2024-04-23T18:33:42.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.825+0000 7fa2fa59c640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa2e40a3bb0 0x7fa2e40a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4000f80 tx=0x7fa2f4005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:42.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:42.829+0000 7fa2eb7fe640 1 -- 172.21.15.132:0/4072300593 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2ec09d320 con 0x7fa2fc103b50 2024-04-23T18:33:43.004 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:33:42 smithi148 podman[62202]: 2024-04-23 18:33:42.637858572 +0000 UTC m=+0.018040894 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:33:43.004 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:33:42 smithi148 podman[62202]: 2024-04-23 18:33:42.752071493 +0000 UTC m=+0.132253780 container create a50d964884fa0352d4e1cf3cdcc077390deeb76721879fbee02c64d5d713c783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-23T18:33:43.005 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:33:43 smithi148 podman[62202]: 2024-04-23 18:33:43.013015083 +0000 UTC m=+0.393197370 container init a50d964884fa0352d4e1cf3cdcc077390deeb76721879fbee02c64d5d713c783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-23T18:33:43.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.041+0000 7fa300907640 1 -- 172.21.15.132:0/4072300593 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2fc062b50 con 0x7fa2fc103b50 2024-04-23T18:33:43.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.042+0000 7fa2eb7fe640 1 -- 172.21.15.132:0/4072300593 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7fa2ec087090 con 0x7fa2fc103b50 2024-04-23T18:33:43.034 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:43.034 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T18:33:43.035 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 8 2024-04-23T18:33:43.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.044+0000 7fa300907640 1 -- 172.21.15.132:0/4072300593 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa2e40a3bb0 msgr2=0x7fa2e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:43.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.044+0000 7fa300907640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa2e40a3bb0 0x7fa2e40a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4000f80 tx=0x7fa2f4005ef0 comp rx=0 tx=0).stop 2024-04-23T18:33:43.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.044+0000 7fa300907640 1 -- 172.21.15.132:0/4072300593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2fc103b50 msgr2=0x7fa2fc19f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:43.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.044+0000 7fa300907640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2fc103b50 0x7fa2fc19f7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00e970 tx=0x7fa2ec00ed70 comp rx=0 tx=0).stop 2024-04-23T18:33:43.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.044+0000 7fa300907640 1 -- 172.21.15.132:0/4072300593 shutdown_connections 2024-04-23T18:33:43.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.044+0000 7fa300907640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa2e40a3bb0 0x7fa2e40a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:43.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.044+0000 7fa300907640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2fc103b50 0x7fa2fc19f7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:43.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.044+0000 7fa300907640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2fc103190 0x7fa2fc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:43.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.044+0000 7fa300907640 1 --2- 172.21.15.132:0/4072300593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2fc101fa0 0x7fa2fc1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:43.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.044+0000 7fa300907640 1 -- 172.21.15.132:0/4072300593 >> 172.21.15.132:0/4072300593 conn(0x7fa2fc0fd530 msgr2=0x7fa2fc1064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:43.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.044+0000 7fa300907640 1 -- 172.21.15.132:0/4072300593 shutdown_connections 2024-04-23T18:33:43.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:43.044+0000 7fa300907640 1 -- 172.21.15.132:0/4072300593 wait complete. 2024-04-23T18:33:43.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:43 smithi148 ceph-mon[32311]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:43.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:43 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/4072300593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:43.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:43 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:43.265 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:43 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:43.265 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:33:43 smithi148 podman[62202]: 2024-04-23 18:33:43.015927036 +0000 UTC m=+0.396109325 container start a50d964884fa0352d4e1cf3cdcc077390deeb76721879fbee02c64d5d713c783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:33:43.265 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:33:43 smithi148 ceph-mds[62221]: -- 172.21.15.148:0/3716758023 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a7875e7a40 con 0x55a7875ba800 2024-04-23T18:33:43.265 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:33:43 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-h[62217]: starting mds.h at 2024-04-23T18:33:43.265 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:33:43 smithi148 bash[62202]: a50d964884fa0352d4e1cf3cdcc077390deeb76721879fbee02c64d5d713c783 2024-04-23T18:33:43.265 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:33:43 smithi148 systemd[1]: Started Ceph mds.h for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:33:43.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:43 smithi132 ceph-mon[29178]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:43.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:43 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/4072300593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:43.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:43 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:43.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:43 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:43.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:43 smithi157 ceph-mon[32580]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:43.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:43 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/4072300593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:43.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:43 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:43.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:43 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:44.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:44 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] up:boot 2024-04-23T18:33:44.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:44 smithi148 ceph-mon[32311]: fsmap 8 up:standby 2024-04-23T18:33:44.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:44 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:33:44.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:44 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:44.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:44 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:44.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:44 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:44.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:44 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:44.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:44 smithi148 ceph-mon[32311]: Deploying daemon mds.i on smithi157 2024-04-23T18:33:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:44 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] up:boot 2024-04-23T18:33:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:44 smithi157 ceph-mon[32580]: fsmap 8 up:standby 2024-04-23T18:33:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:33:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:44 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:44.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:44 smithi157 ceph-mon[32580]: Deploying daemon mds.i on smithi157 2024-04-23T18:33:44.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:44 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] up:boot 2024-04-23T18:33:44.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:44 smithi132 ceph-mon[29178]: fsmap 8 up:standby 2024-04-23T18:33:44.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:33:44.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:44.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:44.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:44.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:44 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:44.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:44 smithi132 ceph-mon[29178]: Deploying daemon mds.i on smithi157 2024-04-23T18:33:45.048 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:45.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 -- 172.21.15.132:0/747046207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d4100e90 msgr2=0x7ff8d41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:45.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 --2- 172.21.15.132:0/747046207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d4100e90 0x7ff8d41012b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d002a410 comp rx=0 tx=0).stop 2024-04-23T18:33:45.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 -- 172.21.15.132:0/747046207 shutdown_connections 2024-04-23T18:33:45.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 --2- 172.21.15.132:0/747046207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8d4101fe0 0x7ff8d4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:45.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 --2- 172.21.15.132:0/747046207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8d41029a0 0x7ff8d41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:45.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 --2- 172.21.15.132:0/747046207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d4100e90 0x7ff8d41012b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 -- 172.21.15.132:0/747046207 >> 172.21.15.132:0/747046207 conn(0x7ff8d40fc400 msgr2=0x7ff8d40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 -- 172.21.15.132:0/747046207 shutdown_connections 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 -- 172.21.15.132:0/747046207 wait complete. 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 Processor -- start 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 -- start start 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8d4100e90 0x7ff8d41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.166+0000 7ff8db38e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8d4101fe0 0x7ff8d4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8db38e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d41029a0 0x7ff8d419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8db38e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d410daf0 con 0x7ff8d41029a0 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8db38e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d410d970 con 0x7ff8d4100e90 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8db38e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d410dc70 con 0x7ff8d4101fe0 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8d992b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d41029a0 0x7ff8d419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8d992b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d41029a0 0x7ff8d419f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55382/0 (socket says 172.21.15.132:55382) 2024-04-23T18:33:45.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8d992b640 1 -- 172.21.15.132:0/2065167836 learned_addr learned my addr 172.21.15.132:0/2065167836 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8d912a640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8d4100e90 0x7ff8d41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8d8929640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8d4101fe0 0x7ff8d4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8d992b640 1 -- 172.21.15.132:0/2065167836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8d4101fe0 msgr2=0x7ff8d4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8d992b640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8d4101fe0 0x7ff8d4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8d992b640 1 -- 172.21.15.132:0/2065167836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8d4100e90 msgr2=0x7ff8d41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8d992b640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8d4100e90 0x7ff8d41981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8d992b640 1 -- 172.21.15.132:0/2065167836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d419fe10 con 0x7ff8d41029a0 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8d992b640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d41029a0 0x7ff8d419f7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00eaa0 tx=0x7ff8cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8c67fc640 1 -- 172.21.15.132:0/2065167836 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc00cdb0 con 0x7ff8d41029a0 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8c67fc640 1 -- 172.21.15.132:0/2065167836 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8cc005020 con 0x7ff8d41029a0 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8db38e640 1 -- 172.21.15.132:0/2065167836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d41a0100 con 0x7ff8d41029a0 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8c67fc640 1 -- 172.21.15.132:0/2065167836 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc0116d0 con 0x7ff8d41029a0 2024-04-23T18:33:45.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.167+0000 7ff8db38e640 1 -- 172.21.15.132:0/2065167836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d41a0650 con 0x7ff8d41029a0 2024-04-23T18:33:45.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.168+0000 7ff8db38e640 1 -- 172.21.15.132:0/2065167836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d4108270 con 0x7ff8d41029a0 2024-04-23T18:33:45.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.169+0000 7ff8c67fc640 1 -- 172.21.15.132:0/2065167836 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff8cc0051d0 con 0x7ff8d41029a0 2024-04-23T18:33:45.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.170+0000 7ff8c67fc640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff8c00a3c30 0x7ff8c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:45.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.170+0000 7ff8d912a640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff8c00a3c30 0x7ff8c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:45.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.171+0000 7ff8c67fc640 1 -- 172.21.15.132:0/2065167836 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff8cc0d04e0 con 0x7ff8d41029a0 2024-04-23T18:33:45.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.171+0000 7ff8d912a640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff8c00a3c30 0x7ff8c00a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0000c00 tx=0x7ff8d00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:45.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.174+0000 7ff8c67fc640 1 -- 172.21.15.132:0/2065167836 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8cc016030 con 0x7ff8d41029a0 2024-04-23T18:33:45.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:45 smithi132 ceph-mon[29178]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:45.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.392+0000 7ff8db38e640 1 -- 172.21.15.132:0/2065167836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8d402d5c0 con 0x7ff8d41029a0 2024-04-23T18:33:45.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.393+0000 7ff8c67fc640 1 -- 172.21.15.132:0/2065167836 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7ff8cc09d380 con 0x7ff8d41029a0 2024-04-23T18:33:45.385 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:45.385 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T18:33:45.385 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 9 2024-04-23T18:33:45.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.395+0000 7ff8db38e640 1 -- 172.21.15.132:0/2065167836 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff8c00a3c30 msgr2=0x7ff8c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:45.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.395+0000 7ff8db38e640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff8c00a3c30 0x7ff8c00a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0000c00 tx=0x7ff8d00314a0 comp rx=0 tx=0).stop 2024-04-23T18:33:45.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.395+0000 7ff8db38e640 1 -- 172.21.15.132:0/2065167836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d41029a0 msgr2=0x7ff8d419f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:45.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.395+0000 7ff8db38e640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d41029a0 0x7ff8d419f7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00eaa0 tx=0x7ff8cc00ef60 comp rx=0 tx=0).stop 2024-04-23T18:33:45.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.395+0000 7ff8db38e640 1 -- 172.21.15.132:0/2065167836 shutdown_connections 2024-04-23T18:33:45.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.395+0000 7ff8db38e640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff8c00a3c30 0x7ff8c00a60f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:45.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.395+0000 7ff8db38e640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8d4101fe0 0x7ff8d4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:45.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.395+0000 7ff8db38e640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d41029a0 0x7ff8d419f7a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:45.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.395+0000 7ff8db38e640 1 --2- 172.21.15.132:0/2065167836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8d4100e90 0x7ff8d41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:45.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.395+0000 7ff8db38e640 1 -- 172.21.15.132:0/2065167836 >> 172.21.15.132:0/2065167836 conn(0x7ff8d40fc400 msgr2=0x7ff8d4105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:45.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.395+0000 7ff8db38e640 1 -- 172.21.15.132:0/2065167836 shutdown_connections 2024-04-23T18:33:45.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:45.395+0000 7ff8db38e640 1 -- 172.21.15.132:0/2065167836 wait complete. 2024-04-23T18:33:45.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:45 smithi148 ceph-mon[32311]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:45.532 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:45 smithi157 ceph-mon[32580]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:45.532 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:33:45 smithi157 systemd[1]: Starting Ceph mds.i for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:33:45.790 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:33:45 smithi157 podman[63488]: 2024-04-23 18:33:45.443760153 +0000 UTC m=+0.018460459 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:33:45.790 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:33:45 smithi157 podman[63488]: 2024-04-23 18:33:45.542148287 +0000 UTC m=+0.116848581 container create e3360bcde07088809f315a626ffd9067693ed4e99ba5af3840f2246d90355617 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-23T18:33:46.210 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:33:45 smithi157 podman[63488]: 2024-04-23 18:33:45.812422141 +0000 UTC m=+0.387122433 container init e3360bcde07088809f315a626ffd9067693ed4e99ba5af3840f2246d90355617 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-i, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-23T18:33:46.211 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:33:45 smithi157 podman[63488]: 2024-04-23 18:33:45.815049125 +0000 UTC m=+0.389749417 container start e3360bcde07088809f315a626ffd9067693ed4e99ba5af3840f2246d90355617 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-23T18:33:46.211 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:33:45 smithi157 ceph-mds[63507]: -- 172.21.15.157:0/909604009 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558694eeda40 con 0x558694ec0800 2024-04-23T18:33:46.211 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:33:45 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-i[63503]: starting mds.i at 2024-04-23T18:33:46.211 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:33:45 smithi157 bash[63488]: e3360bcde07088809f315a626ffd9067693ed4e99ba5af3840f2246d90355617 2024-04-23T18:33:46.211 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:33:45 smithi157 systemd[1]: Started Ceph mds.i for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:33:46.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:46 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2065167836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:46.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:46.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:46.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:46 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:46.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:46 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2065167836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:46.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:46.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:46.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:46.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:46.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:46.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:46 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:46.603 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:46 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2065167836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:46.603 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:46.604 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:46.604 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:46.604 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:46.604 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:46.604 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:46 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:47.400 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:47.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:47 smithi148 ceph-mon[32311]: Deploying daemon mds.g on smithi132 2024-04-23T18:33:47.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:47 smithi148 ceph-mon[32311]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:47.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:47 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] up:boot 2024-04-23T18:33:47.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:47 smithi148 ceph-mon[32311]: fsmap 9 up:standby 2024-04-23T18:33:47.464 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:47 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:33:47.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.509+0000 7f620cca7640 1 -- 172.21.15.132:0/1824869226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6208103190 msgr2=0x7f6208103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:47.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.509+0000 7f620cca7640 1 --2- 172.21.15.132:0/1824869226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6208103190 0x7f6208103610 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009ca0 tx=0x7f61fc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:33:47.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.509+0000 7f620cca7640 1 -- 172.21.15.132:0/1824869226 shutdown_connections 2024-04-23T18:33:47.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.509+0000 7f620cca7640 1 --2- 172.21.15.132:0/1824869226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6208103b50 0x7f620810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:47.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.509+0000 7f620cca7640 1 --2- 172.21.15.132:0/1824869226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6208103190 0x7f6208103610 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:47.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.509+0000 7f620cca7640 1 --2- 172.21.15.132:0/1824869226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6208101fa0 0x7f62081023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:47.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.509+0000 7f620cca7640 1 -- 172.21.15.132:0/1824869226 >> 172.21.15.132:0/1824869226 conn(0x7f62080fd530 msgr2=0x7f62080ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:47.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.509+0000 7f620cca7640 1 -- 172.21.15.132:0/1824869226 shutdown_connections 2024-04-23T18:33:47.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.509+0000 7f620cca7640 1 -- 172.21.15.132:0/1824869226 wait complete. 2024-04-23T18:33:47.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.509+0000 7f620cca7640 1 Processor -- start 2024-04-23T18:33:47.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.509+0000 7f620cca7640 1 -- start start 2024-04-23T18:33:47.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f620cca7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6208101fa0 0x7f6208198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:47.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f620cca7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6208103190 0x7f6208198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:47.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f620cca7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6208103b50 0x7f620819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:47.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f620cca7640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f620810db20 con 0x7f6208103190 2024-04-23T18:33:47.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f620cca7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f620810d9a0 con 0x7f6208103b50 2024-04-23T18:33:47.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f620cca7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f620810dca0 con 0x7f6208101fa0 2024-04-23T18:33:47.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f6205d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6208103190 0x7f6208198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:47.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f6205d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6208103190 0x7f6208198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55398/0 (socket says 172.21.15.132:55398) 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f6205d9b640 1 -- 172.21.15.132:0/4073897602 learned_addr learned my addr 172.21.15.132:0/4073897602 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f620659c640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6208101fa0 0x7f6208198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f6206d9d640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6208103b50 0x7f620819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f6205d9b640 1 -- 172.21.15.132:0/4073897602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6208101fa0 msgr2=0x7f6208198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f6205d9b640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6208101fa0 0x7f6208198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f6205d9b640 1 -- 172.21.15.132:0/4073897602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6208103b50 msgr2=0x7f620819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f6205d9b640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6208103b50 0x7f620819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f6205d9b640 1 -- 172.21.15.132:0/4073897602 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620819fe20 con 0x7f6208103190 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f6205d9b640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6208103190 0x7f6208198740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f61fc006010 tx=0x7f61fc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f61f77fe640 1 -- 172.21.15.132:0/4073897602 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0317c0 con 0x7f6208103190 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f61f77fe640 1 -- 172.21.15.132:0/4073897602 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61fc0044f0 con 0x7f6208103190 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f620cca7640 1 -- 172.21.15.132:0/4073897602 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62081a00b0 con 0x7f6208103190 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f61f77fe640 1 -- 172.21.15.132:0/4073897602 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc02fd10 con 0x7f6208103190 2024-04-23T18:33:47.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.510+0000 7f620cca7640 1 -- 172.21.15.132:0/4073897602 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62081a0680 con 0x7f6208103190 2024-04-23T18:33:47.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.512+0000 7f620cca7640 1 -- 172.21.15.132:0/4073897602 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6208109340 con 0x7f6208103190 2024-04-23T18:33:47.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.512+0000 7f61f77fe640 1 -- 172.21.15.132:0/4073897602 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f61fc004040 con 0x7f6208103190 2024-04-23T18:33:47.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.513+0000 7f61f77fe640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f61f00a3bb0 0x7f61f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:47.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.513+0000 7f620659c640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f61f00a3bb0 0x7f61f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:47.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.513+0000 7f61f77fe640 1 -- 172.21.15.132:0/4073897602 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f61fc0f14c0 con 0x7f6208103190 2024-04-23T18:33:47.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.513+0000 7f620659c640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f61f00a3bb0 0x7f61f00a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6208102ff0 tx=0x7f620000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:47.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.517+0000 7f61f77fe640 1 -- 172.21.15.132:0/4073897602 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f61fc0be340 con 0x7f6208103190 2024-04-23T18:33:47.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:47 smithi157 ceph-mon[32580]: Deploying daemon mds.g on smithi132 2024-04-23T18:33:47.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:47 smithi157 ceph-mon[32580]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:47.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:47 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] up:boot 2024-04-23T18:33:47.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:47 smithi157 ceph-mon[32580]: fsmap 9 up:standby 2024-04-23T18:33:47.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:47 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:33:47.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:47 smithi132 ceph-mon[29178]: Deploying daemon mds.g on smithi132 2024-04-23T18:33:47.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:47 smithi132 ceph-mon[29178]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:47.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:47 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] up:boot 2024-04-23T18:33:47.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:47 smithi132 ceph-mon[29178]: fsmap 9 up:standby 2024-04-23T18:33:47.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:47 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:33:47.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.740+0000 7f620cca7640 1 -- 172.21.15.132:0/4073897602 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f620802d770 con 0x7f6208103190 2024-04-23T18:33:47.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.741+0000 7f61f77fe640 1 -- 172.21.15.132:0/4073897602 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7f61fc0bdc40 con 0x7f6208103190 2024-04-23T18:33:47.733 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:47.733 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T18:33:47.734 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 10 2024-04-23T18:33:47.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.743+0000 7f620cca7640 1 -- 172.21.15.132:0/4073897602 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f61f00a3bb0 msgr2=0x7f61f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:47.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.743+0000 7f620cca7640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f61f00a3bb0 0x7f61f00a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6208102ff0 tx=0x7f620000b390 comp rx=0 tx=0).stop 2024-04-23T18:33:47.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.743+0000 7f620cca7640 1 -- 172.21.15.132:0/4073897602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6208103190 msgr2=0x7f6208198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:47.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.743+0000 7f620cca7640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6208103190 0x7f6208198740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f61fc006010 tx=0x7f61fc003ba0 comp rx=0 tx=0).stop 2024-04-23T18:33:47.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.743+0000 7f620cca7640 1 -- 172.21.15.132:0/4073897602 shutdown_connections 2024-04-23T18:33:47.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.743+0000 7f620cca7640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f61f00a3bb0 0x7f61f00a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:47.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.743+0000 7f620cca7640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6208103b50 0x7f620819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:47.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.743+0000 7f620cca7640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6208103190 0x7f6208198740 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:47.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.743+0000 7f620cca7640 1 --2- 172.21.15.132:0/4073897602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6208101fa0 0x7f6208198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:47.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.743+0000 7f620cca7640 1 -- 172.21.15.132:0/4073897602 >> 172.21.15.132:0/4073897602 conn(0x7f62080fd530 msgr2=0x7f6208106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:47.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.743+0000 7f620cca7640 1 -- 172.21.15.132:0/4073897602 shutdown_connections 2024-04-23T18:33:47.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:47.743+0000 7f620cca7640 1 -- 172.21.15.132:0/4073897602 wait complete. 2024-04-23T18:33:48.220 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:33:48 smithi132 systemd[1]: Starting Ceph mds.g for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:33:48.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:48 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/4073897602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:48.478 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:33:48 smithi132 podman[70363]: 2024-04-23 18:33:48.245851311 +0000 UTC m=+0.018291890 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:33:48.478 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:33:48 smithi132 podman[70363]: 2024-04-23 18:33:48.35986611 +0000 UTC m=+0.132306657 container create fcb9452407eab67c9a29a376caef143514176a2d934d830de7e0075c0da10ab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:33:48.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:48 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/4073897602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:48.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:48 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/4073897602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:48.861 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:33:48 smithi132 podman[70363]: 2024-04-23 18:33:48.613029417 +0000 UTC m=+0.385469965 container init fcb9452407eab67c9a29a376caef143514176a2d934d830de7e0075c0da10ab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-23T18:33:48.862 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:33:48 smithi132 podman[70363]: 2024-04-23 18:33:48.615947736 +0000 UTC m=+0.388388283 container start fcb9452407eab67c9a29a376caef143514176a2d934d830de7e0075c0da10ab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:33:48.862 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:33:48 smithi132 ceph-mds[70383]: -- 172.21.15.132:0/1580269746 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a135ccba40 con 0x55a135c9ec00 2024-04-23T18:33:48.862 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:33:48 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-g[70379]: starting mds.g at 2024-04-23T18:33:48.862 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:33:48 smithi132 bash[70363]: fcb9452407eab67c9a29a376caef143514176a2d934d830de7e0075c0da10ab4 2024-04-23T18:33:48.862 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:33:48 smithi132 systemd[1]: Started Ceph mds.g for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:33:49.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:49 smithi148 ceph-mon[32311]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:49.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:49.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:49 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:49.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:49 smithi157 ceph-mon[32580]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:49.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:49.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:49 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:49.566 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:49 smithi132 ceph-mon[29178]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:49.566 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:49.566 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:49.566 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:49.566 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:49.567 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:49.567 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:49 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:49.747 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:49.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 -- 172.21.15.132:0/9515310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b4c101fa0 msgr2=0x7f3b4c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:49.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 --2- 172.21.15.132:0/9515310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b4c101fa0 0x7f3b4c1023c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009900 tx=0x7f3b3c02a410 comp rx=0 tx=0).stop 2024-04-23T18:33:49.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 -- 172.21.15.132:0/9515310 shutdown_connections 2024-04-23T18:33:49.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 --2- 172.21.15.132:0/9515310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b4c103190 0x7f3b4c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:49.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 --2- 172.21.15.132:0/9515310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b4c103b50 0x7f3b4c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 --2- 172.21.15.132:0/9515310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b4c101fa0 0x7f3b4c1023c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 -- 172.21.15.132:0/9515310 >> 172.21.15.132:0/9515310 conn(0x7f3b4c0fd550 msgr2=0x7f3b4c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 -- 172.21.15.132:0/9515310 shutdown_connections 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 -- 172.21.15.132:0/9515310 wait complete. 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 Processor -- start 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 -- start start 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b4c101fa0 0x7f3b4c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b4c103190 0x7f3b4c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b4c103b50 0x7f3b4c19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c10dba0 con 0x7f3b4c103190 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c10da20 con 0x7f3b4c103b50 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b51a64640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c10dd20 con 0x7f3b4c101fa0 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b43fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b4c103190 0x7f3b4c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b43fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b4c103190 0x7f3b4c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43768/0 (socket says 172.21.15.132:43768) 2024-04-23T18:33:49.852 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.856+0000 7f3b43fff640 1 -- 172.21.15.132:0/167262624 learned_addr learned my addr 172.21.15.132:0/167262624 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b4bfff640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b4c103b50 0x7f3b4c19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b4b7fe640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b4c101fa0 0x7f3b4c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b43fff640 1 -- 172.21.15.132:0/167262624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b4c101fa0 msgr2=0x7f3b4c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b43fff640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b4c101fa0 0x7f3b4c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b43fff640 1 -- 172.21.15.132:0/167262624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b4c103b50 msgr2=0x7f3b4c19f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b43fff640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b4c103b50 0x7f3b4c19f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b43fff640 1 -- 172.21.15.132:0/167262624 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c19ffb0 con 0x7f3b4c103190 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b4b7fe640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b4c101fa0 0x7f3b4c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b43fff640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b4c103190 0x7f3b4c1988d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400b4a0 tx=0x7f3b3400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b497fa640 1 -- 172.21.15.132:0/167262624 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34014070 con 0x7f3b4c103190 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b497fa640 1 -- 172.21.15.132:0/167262624 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b34010ac0 con 0x7f3b4c103190 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b497fa640 1 -- 172.21.15.132:0/167262624 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34017c30 con 0x7f3b4c103190 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b51a64640 1 -- 172.21.15.132:0/167262624 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c1a02a0 con 0x7f3b4c103190 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.857+0000 7f3b51a64640 1 -- 172.21.15.132:0/167262624 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c1a0820 con 0x7f3b4c103190 2024-04-23T18:33:49.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.858+0000 7f3b51a64640 1 -- 172.21.15.132:0/167262624 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c109480 con 0x7f3b4c103190 2024-04-23T18:33:49.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.859+0000 7f3b497fa640 1 -- 172.21.15.132:0/167262624 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f3b34017070 con 0x7f3b4c103190 2024-04-23T18:33:49.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.860+0000 7f3b497fa640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3b380a3bb0 0x7f3b380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:49.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.860+0000 7f3b497fa640 1 -- 172.21.15.132:0/167262624 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3b340cf6f0 con 0x7f3b4c103190 2024-04-23T18:33:49.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.860+0000 7f3b4b7fe640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3b380a3bb0 0x7f3b380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:49.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.860+0000 7f3b4b7fe640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3b380a3bb0 0x7f3b380a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c000c00 tx=0x7f3b3c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:49.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:49.863+0000 7f3b497fa640 1 -- 172.21.15.132:0/167262624 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b3409c570 con 0x7f3b4c103190 2024-04-23T18:33:50.076 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.083+0000 7f3b51a64640 1 -- 172.21.15.132:0/167262624 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b4c02d810 con 0x7f3b4c103190 2024-04-23T18:33:50.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.084+0000 7f3b497fa640 1 -- 172.21.15.132:0/167262624 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7f3b3409be70 con 0x7f3b4c103190 2024-04-23T18:33:50.077 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:50.078 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T18:33:50.078 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 11 2024-04-23T18:33:50.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.086+0000 7f3b51a64640 1 -- 172.21.15.132:0/167262624 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3b380a3bb0 msgr2=0x7f3b380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:50.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.086+0000 7f3b51a64640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3b380a3bb0 0x7f3b380a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c000c00 tx=0x7f3b3c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:33:50.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.086+0000 7f3b51a64640 1 -- 172.21.15.132:0/167262624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b4c103190 msgr2=0x7f3b4c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:50.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.086+0000 7f3b51a64640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b4c103190 0x7f3b4c1988d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400b4a0 tx=0x7f3b3400b960 comp rx=0 tx=0).stop 2024-04-23T18:33:50.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.086+0000 7f3b51a64640 1 -- 172.21.15.132:0/167262624 shutdown_connections 2024-04-23T18:33:50.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.086+0000 7f3b51a64640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3b380a3bb0 0x7f3b380a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:50.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.086+0000 7f3b51a64640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b4c103b50 0x7f3b4c19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:50.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.086+0000 7f3b51a64640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b4c103190 0x7f3b4c1988d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:50.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.086+0000 7f3b51a64640 1 --2- 172.21.15.132:0/167262624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b4c101fa0 0x7f3b4c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:50.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.086+0000 7f3b51a64640 1 -- 172.21.15.132:0/167262624 >> 172.21.15.132:0/167262624 conn(0x7f3b4c0fd550 msgr2=0x7f3b4c106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:50.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.086+0000 7f3b51a64640 1 -- 172.21.15.132:0/167262624 shutdown_connections 2024-04-23T18:33:50.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:50.086+0000 7f3b51a64640 1 -- 172.21.15.132:0/167262624 wait complete. 2024-04-23T18:33:50.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:50 smithi132 ceph-mon[29178]: Deploying daemon mds.a on smithi132 2024-04-23T18:33:50.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:50 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.132:6838/579251026,v1:172.21.15.132:6839/579251026] up:boot 2024-04-23T18:33:50.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:50 smithi132 ceph-mon[29178]: fsmap 10 up:standby 2024-04-23T18:33:50.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:50 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:33:50.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:50 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/167262624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:50.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:50 smithi148 ceph-mon[32311]: Deploying daemon mds.a on smithi132 2024-04-23T18:33:50.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:50 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.132:6838/579251026,v1:172.21.15.132:6839/579251026] up:boot 2024-04-23T18:33:50.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:50 smithi148 ceph-mon[32311]: fsmap 10 up:standby 2024-04-23T18:33:50.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:50 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:33:50.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:50 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/167262624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:50.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:50 smithi157 ceph-mon[32580]: Deploying daemon mds.a on smithi132 2024-04-23T18:33:50.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:50 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.132:6838/579251026,v1:172.21.15.132:6839/579251026] up:boot 2024-04-23T18:33:50.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:50 smithi157 ceph-mon[32580]: fsmap 10 up:standby 2024-04-23T18:33:50.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:50 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:33:50.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:50 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/167262624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:51.112 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:33:50 smithi132 systemd[1]: Starting Ceph mds.a for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:33:51.461 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:51 smithi132 ceph-mon[29178]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:51.461 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:33:51 smithi132 podman[70720]: 2024-04-23 18:33:51.077028164 +0000 UTC m=+0.018351287 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:33:51.461 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:33:51 smithi132 podman[70720]: 2024-04-23 18:33:51.199960309 +0000 UTC m=+0.141283447 container create d826cab44e48fdb52bccce1c7e17724eb9c10c606cea554310e7a62c76b7b4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-23T18:33:51.461 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:33:51 smithi132 podman[70720]: 2024-04-23 18:33:51.469523584 +0000 UTC m=+0.410846711 container init d826cab44e48fdb52bccce1c7e17724eb9c10c606cea554310e7a62c76b7b4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-23T18:33:51.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:51 smithi148 ceph-mon[32311]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:51.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:51 smithi157 ceph-mon[32580]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:51.861 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:33:51 smithi132 podman[70720]: 2024-04-23 18:33:51.472438521 +0000 UTC m=+0.413761654 container start d826cab44e48fdb52bccce1c7e17724eb9c10c606cea554310e7a62c76b7b4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-23T18:33:51.861 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:33:51 smithi132 ceph-mds[70740]: -- 172.21.15.132:0/856060455 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e33df7a40 con 0x556e33dcb000 2024-04-23T18:33:51.862 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:33:51 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-a[70735]: starting mds.a at 2024-04-23T18:33:51.862 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:33:51 smithi132 bash[70720]: d826cab44e48fdb52bccce1c7e17724eb9c10c606cea554310e7a62c76b7b4b0 2024-04-23T18:33:51.862 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:33:51 smithi132 systemd[1]: Started Ceph mds.a for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:33:52.091 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:52.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.204+0000 7f3279f94640 1 -- 172.21.15.132:0/4037722058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3274101b30 msgr2=0x7f32741093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.204+0000 7f3279f94640 1 --2- 172.21.15.132:0/4037722058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3274101b30 0x7f32741093d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f326c009ca0 tx=0x7f326c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.204+0000 7f3279f94640 1 -- 172.21.15.132:0/4037722058 shutdown_connections 2024-04-23T18:33:52.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.204+0000 7f3279f94640 1 --2- 172.21.15.132:0/4037722058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3274101b30 0x7f32741093d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.204+0000 7f3279f94640 1 --2- 172.21.15.132:0/4037722058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3274101170 0x7f32741015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.204+0000 7f3279f94640 1 --2- 172.21.15.132:0/4037722058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3274102440 0x7f3274102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.204+0000 7f3279f94640 1 -- 172.21.15.132:0/4037722058 >> 172.21.15.132:0/4037722058 conn(0x7f32740fbbc0 msgr2=0x7f32740fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:52.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.204+0000 7f3279f94640 1 -- 172.21.15.132:0/4037722058 shutdown_connections 2024-04-23T18:33:52.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.204+0000 7f3279f94640 1 -- 172.21.15.132:0/4037722058 wait complete. 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.204+0000 7f3279f94640 1 Processor -- start 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3279f94640 1 -- start start 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3279f94640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3274101170 0x7f3274198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3279f94640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3274101b30 0x7f32741986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3279f94640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3274102440 0x7f327419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3279f94640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f327410db70 con 0x7f3274101b30 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3279f94640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f327410d9f0 con 0x7f3274102440 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3279f94640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f327410dcf0 con 0x7f3274101170 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3272ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3274101b30 0x7f32741986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3272ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3274101b30 0x7f32741986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43806/0 (socket says 172.21.15.132:43806) 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3272ffd640 1 -- 172.21.15.132:0/504664697 learned_addr learned my addr 172.21.15.132:0/504664697 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3273fff640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3274102440 0x7f327419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f32737fe640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3274101170 0x7f3274198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3272ffd640 1 -- 172.21.15.132:0/504664697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3274101170 msgr2=0x7f3274198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3272ffd640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3274101170 0x7f3274198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3272ffd640 1 -- 172.21.15.132:0/504664697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3274102440 msgr2=0x7f327419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3272ffd640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3274102440 0x7f327419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3272ffd640 1 -- 172.21.15.132:0/504664697 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327419fda0 con 0x7f3274101b30 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3273fff640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3274102440 0x7f327419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3272ffd640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3274101b30 0x7f32741986c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f326400c920 tx=0x7f326400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.205+0000 7f3270ff9640 1 -- 172.21.15.132:0/504664697 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264007c70 con 0x7f3274101b30 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.206+0000 7f3279f94640 1 -- 172.21.15.132:0/504664697 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32741a0090 con 0x7f3274101b30 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.206+0000 7f3270ff9640 1 -- 172.21.15.132:0/504664697 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3264007e10 con 0x7f3274101b30 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.206+0000 7f3270ff9640 1 -- 172.21.15.132:0/504664697 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264018820 con 0x7f3274101b30 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.206+0000 7f3279f94640 1 -- 172.21.15.132:0/504664697 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32741a06b0 con 0x7f3274101b30 2024-04-23T18:33:52.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.207+0000 7f3279f94640 1 -- 172.21.15.132:0/504664697 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32741082b0 con 0x7f3274101b30 2024-04-23T18:33:52.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.208+0000 7f3270ff9640 1 -- 172.21.15.132:0/504664697 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f326400f450 con 0x7f3274101b30 2024-04-23T18:33:52.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.208+0000 7f3270ff9640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f325c0a3bb0 0x7f325c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.208+0000 7f32737fe640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f325c0a3bb0 0x7f325c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.208+0000 7f3270ff9640 1 -- 172.21.15.132:0/504664697 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f32640d0120 con 0x7f3274101b30 2024-04-23T18:33:52.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.209+0000 7f32737fe640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f325c0a3bb0 0x7f325c0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f327406a350 tx=0x7f3268008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:52.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.213+0000 7f3270ff9640 1 -- 172.21.15.132:0/504664697 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3264014030 con 0x7f3274101b30 2024-04-23T18:33:52.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.429+0000 7f3279f94640 1 -- 172.21.15.132:0/504664697 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f327402d7c0 con 0x7f3274101b30 2024-04-23T18:33:52.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.430+0000 7f3270ff9640 1 -- 172.21.15.132:0/504664697 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7f326409cfa0 con 0x7f3274101b30 2024-04-23T18:33:52.423 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:52.423 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T18:33:52.423 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 12 2024-04-23T18:33:52.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.432+0000 7f3279f94640 1 -- 172.21.15.132:0/504664697 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f325c0a3bb0 msgr2=0x7f325c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.432+0000 7f3279f94640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f325c0a3bb0 0x7f325c0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f327406a350 tx=0x7f3268008040 comp rx=0 tx=0).stop 2024-04-23T18:33:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.432+0000 7f3279f94640 1 -- 172.21.15.132:0/504664697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3274101b30 msgr2=0x7f32741986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.432+0000 7f3279f94640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3274101b30 0x7f32741986c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f326400c920 tx=0x7f326400cde0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.432+0000 7f3279f94640 1 -- 172.21.15.132:0/504664697 shutdown_connections 2024-04-23T18:33:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.432+0000 7f3279f94640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f325c0a3bb0 0x7f325c0a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.432+0000 7f3279f94640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3274102440 0x7f327419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.432+0000 7f3279f94640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3274101b30 0x7f32741986c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.432+0000 7f3279f94640 1 --2- 172.21.15.132:0/504664697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3274101170 0x7f3274198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.432+0000 7f3279f94640 1 -- 172.21.15.132:0/504664697 >> 172.21.15.132:0/504664697 conn(0x7f32740fbbc0 msgr2=0x7f3274105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.432+0000 7f3279f94640 1 -- 172.21.15.132:0/504664697 shutdown_connections 2024-04-23T18:33:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:52.432+0000 7f3279f94640 1 -- 172.21.15.132:0/504664697 wait complete. 2024-04-23T18:33:52.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:52 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:52.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:52 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:52.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:52 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:52.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:52 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:52.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:52 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:52.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:52 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:52.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:52 smithi132 ceph-mon[29178]: Deploying daemon mds.b on smithi148 2024-04-23T18:33:52.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:52 smithi132 ceph-mon[29178]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:52.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:52 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.132:6840/826931131,v1:172.21.15.132:6841/826931131] up:boot 2024-04-23T18:33:52.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:52 smithi132 ceph-mon[29178]: fsmap 11 up:standby 2024-04-23T18:33:52.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:52 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:33:52.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:52 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/504664697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:53.004 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:52 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:53.004 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:52 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:53.004 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:52 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:53.004 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:52 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:53.004 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:52 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:53.004 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:52 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:53.004 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:52 smithi148 ceph-mon[32311]: Deploying daemon mds.b on smithi148 2024-04-23T18:33:53.005 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:52 smithi148 ceph-mon[32311]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:53.005 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:52 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.132:6840/826931131,v1:172.21.15.132:6841/826931131] up:boot 2024-04-23T18:33:53.005 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:52 smithi148 ceph-mon[32311]: fsmap 11 up:standby 2024-04-23T18:33:53.005 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:52 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:33:53.005 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:52 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/504664697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:53.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:52 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:53.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:52 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:53.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:52 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:53.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:52 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:33:53.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:52 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:33:53.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:52 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:53.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:52 smithi157 ceph-mon[32580]: Deploying daemon mds.b on smithi148 2024-04-23T18:33:53.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:52 smithi157 ceph-mon[32580]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:53.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:52 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.132:6840/826931131,v1:172.21.15.132:6841/826931131] up:boot 2024-04-23T18:33:53.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:52 smithi157 ceph-mon[32580]: fsmap 11 up:standby 2024-04-23T18:33:53.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:52 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:33:53.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:52 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/504664697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:54.184 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:33:53 smithi148 systemd[1]: Starting Ceph mds.b for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:33:54.437 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:54.484 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:33:54 smithi148 podman[62510]: 2024-04-23 18:33:54.095207983 +0000 UTC m=+0.018099634 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:33:54.484 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:33:54 smithi148 podman[62510]: 2024-04-23 18:33:54.223481737 +0000 UTC m=+0.146373377 container create 9219f4346f4cc423ed2e410393351956263eecf7f2f685bc37f708328077775c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-23T18:33:54.485 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:33:54 smithi148 podman[62510]: 2024-04-23 18:33:54.493168753 +0000 UTC m=+0.416060394 container init 9219f4346f4cc423ed2e410393351956263eecf7f2f685bc37f708328077775c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T18:33:54.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.556+0000 7fe9eca67640 1 -- 172.21.15.132:0/3660140957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9e8101ff0 msgr2=0x7fe9e8102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:54.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.556+0000 7fe9eca67640 1 --2- 172.21.15.132:0/3660140957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9e8101ff0 0x7fe9e8102470 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009900 tx=0x7fe9d802a410 comp rx=0 tx=0).stop 2024-04-23T18:33:54.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.556+0000 7fe9eca67640 1 -- 172.21.15.132:0/3660140957 shutdown_connections 2024-04-23T18:33:54.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.556+0000 7fe9eca67640 1 --2- 172.21.15.132:0/3660140957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e81029b0 0x7fe9e81092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:54.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.556+0000 7fe9eca67640 1 --2- 172.21.15.132:0/3660140957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9e8101ff0 0x7fe9e8102470 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:54.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.556+0000 7fe9eca67640 1 --2- 172.21.15.132:0/3660140957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e8100ea0 0x7fe9e81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.556+0000 7fe9eca67640 1 -- 172.21.15.132:0/3660140957 >> 172.21.15.132:0/3660140957 conn(0x7fe9e80fc430 msgr2=0x7fe9e80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.556+0000 7fe9eca67640 1 -- 172.21.15.132:0/3660140957 shutdown_connections 2024-04-23T18:33:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.556+0000 7fe9eca67640 1 -- 172.21.15.132:0/3660140957 wait complete. 2024-04-23T18:33:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9eca67640 1 Processor -- start 2024-04-23T18:33:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9eca67640 1 -- start start 2024-04-23T18:33:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9eca67640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9e8100ea0 0x7fe9e8198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9eca67640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e8101ff0 0x7fe9e8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9eca67640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e81029b0 0x7fe9e819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9eca67640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e810c9e0 con 0x7fe9e8100ea0 2024-04-23T18:33:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9eca67640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e810c860 con 0x7fe9e81029b0 2024-04-23T18:33:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9eca67640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e810cb60 con 0x7fe9e8101ff0 2024-04-23T18:33:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9e659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9e8100ea0 0x7fe9e8198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9e659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9e8100ea0 0x7fe9e8198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43822/0 (socket says 172.21.15.132:43822) 2024-04-23T18:33:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9e659c640 1 -- 172.21.15.132:0/2251532608 learned_addr learned my addr 172.21.15.132:0/2251532608 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9e6d9d640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e81029b0 0x7fe9e819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9e5d9b640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e8101ff0 0x7fe9e8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9e659c640 1 -- 172.21.15.132:0/2251532608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e8101ff0 msgr2=0x7fe9e8198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9e659c640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e8101ff0 0x7fe9e8198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9e659c640 1 -- 172.21.15.132:0/2251532608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e81029b0 msgr2=0x7fe9e819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9e659c640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e81029b0 0x7fe9e819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9e659c640 1 -- 172.21.15.132:0/2251532608 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e819fe20 con 0x7fe9e8100ea0 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9e659c640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9e8100ea0 0x7fe9e8198200 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00e9f0 tx=0x7fe9dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9d77fe640 1 -- 172.21.15.132:0/2251532608 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc00cd80 con 0x7fe9e8100ea0 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9d77fe640 1 -- 172.21.15.132:0/2251532608 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9dc0108e0 con 0x7fe9e8100ea0 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.557+0000 7fe9eca67640 1 -- 172.21.15.132:0/2251532608 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e81a0110 con 0x7fe9e8100ea0 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.558+0000 7fe9d77fe640 1 -- 172.21.15.132:0/2251532608 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc018980 con 0x7fe9e8100ea0 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.558+0000 7fe9eca67640 1 -- 172.21.15.132:0/2251532608 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e81a0630 con 0x7fe9e8100ea0 2024-04-23T18:33:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.559+0000 7fe9eca67640 1 -- 172.21.15.132:0/2251532608 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e81a0890 con 0x7fe9e8100ea0 2024-04-23T18:33:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.560+0000 7fe9d77fe640 1 -- 172.21.15.132:0/2251532608 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe9dc010430 con 0x7fe9e8100ea0 2024-04-23T18:33:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.560+0000 7fe9d77fe640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe9d00a3bb0 0x7fe9d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.560+0000 7fe9e5d9b640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe9d00a3bb0 0x7fe9d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:54.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.560+0000 7fe9d77fe640 1 -- 172.21.15.132:0/2251532608 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe9dc0d0730 con 0x7fe9e8100ea0 2024-04-23T18:33:54.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.561+0000 7fe9e5d9b640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe9d00a3bb0 0x7fe9d00a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009230 tx=0x7fe9d80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:54.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.564+0000 7fe9d77fe640 1 -- 172.21.15.132:0/2251532608 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9dc09cf60 con 0x7fe9e8100ea0 2024-04-23T18:33:54.764 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:33:54 smithi148 podman[62510]: 2024-04-23 18:33:54.49604004 +0000 UTC m=+0.418931691 container start 9219f4346f4cc423ed2e410393351956263eecf7f2f685bc37f708328077775c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-23T18:33:54.764 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:33:54 smithi148 ceph-mds[62529]: -- 172.21.15.148:0/1904804462 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55645d55ba40 con 0x55645d52ec00 2024-04-23T18:33:54.764 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:33:54 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b[62525]: starting mds.b at 2024-04-23T18:33:54.764 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:33:54 smithi148 bash[62510]: 9219f4346f4cc423ed2e410393351956263eecf7f2f685bc37f708328077775c 2024-04-23T18:33:54.764 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:33:54 smithi148 systemd[1]: Started Ceph mds.b for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:33:54.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.791+0000 7fe9eca67640 1 -- 172.21.15.132:0/2251532608 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9e81a0a60 con 0x7fe9e8100ea0 2024-04-23T18:33:54.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.791+0000 7fe9d77fe640 1 -- 172.21.15.132:0/2251532608 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7fe9dc0a1260 con 0x7fe9e8100ea0 2024-04-23T18:33:54.784 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:54.784 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T18:33:54.784 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 12 2024-04-23T18:33:54.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.793+0000 7fe9eca67640 1 -- 172.21.15.132:0/2251532608 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe9d00a3bb0 msgr2=0x7fe9d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:54.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.793+0000 7fe9eca67640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe9d00a3bb0 0x7fe9d00a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009230 tx=0x7fe9d80314a0 comp rx=0 tx=0).stop 2024-04-23T18:33:54.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.793+0000 7fe9eca67640 1 -- 172.21.15.132:0/2251532608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9e8100ea0 msgr2=0x7fe9e8198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:54.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.793+0000 7fe9eca67640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9e8100ea0 0x7fe9e8198200 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00e9f0 tx=0x7fe9dc00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:33:54.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.794+0000 7fe9eca67640 1 -- 172.21.15.132:0/2251532608 shutdown_connections 2024-04-23T18:33:54.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.794+0000 7fe9eca67640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe9d00a3bb0 0x7fe9d00a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:54.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.794+0000 7fe9eca67640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e8101ff0 0x7fe9e8198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:54.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.794+0000 7fe9eca67640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e81029b0 0x7fe9e819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:54.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.794+0000 7fe9eca67640 1 --2- 172.21.15.132:0/2251532608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9e8100ea0 0x7fe9e8198200 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:54.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.794+0000 7fe9eca67640 1 -- 172.21.15.132:0/2251532608 >> 172.21.15.132:0/2251532608 conn(0x7fe9e80fc430 msgr2=0x7fe9e80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:54.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.794+0000 7fe9eca67640 1 -- 172.21.15.132:0/2251532608 shutdown_connections 2024-04-23T18:33:54.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:54.794+0000 7fe9eca67640 1 -- 172.21.15.132:0/2251532608 wait complete. 2024-04-23T18:33:55.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:55 smithi148 ceph-mon[32311]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:55.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:55 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2251532608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:55.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:55 smithi157 ceph-mon[32580]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:55.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:55 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2251532608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:55.583 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:55 smithi132 ceph-mon[29178]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:55.583 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.583 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.583 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.583 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.583 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:55.583 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:55 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2251532608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:55.583 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:56.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:56 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] up:boot 2024-04-23T18:33:56.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:56 smithi148 ceph-mon[32311]: fsmap 12 up:standby 2024-04-23T18:33:56.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:56 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:33:56.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:56 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] up:boot 2024-04-23T18:33:56.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:56 smithi157 ceph-mon[32580]: fsmap 12 up:standby 2024-04-23T18:33:56.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:56 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:33:56.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:56 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] up:boot 2024-04-23T18:33:56.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:56 smithi132 ceph-mon[29178]: fsmap 12 up:standby 2024-04-23T18:33:56.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:56 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:33:56.798 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:56.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.906+0000 7fda90c20640 1 -- 172.21.15.132:0/2094959231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c103190 msgr2=0x7fda8c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:56.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.906+0000 7fda90c20640 1 --2- 172.21.15.132:0/2094959231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c103190 0x7fda8c103610 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fda80009900 tx=0x7fda8002a410 comp rx=0 tx=0).stop 2024-04-23T18:33:56.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.906+0000 7fda90c20640 1 -- 172.21.15.132:0/2094959231 shutdown_connections 2024-04-23T18:33:56.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.906+0000 7fda90c20640 1 --2- 172.21.15.132:0/2094959231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda8c103b50 0x7fda8c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:56.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.906+0000 7fda90c20640 1 --2- 172.21.15.132:0/2094959231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c103190 0x7fda8c103610 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:56.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.906+0000 7fda90c20640 1 --2- 172.21.15.132:0/2094959231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda8c101fa0 0x7fda8c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.906+0000 7fda90c20640 1 -- 172.21.15.132:0/2094959231 >> 172.21.15.132:0/2094959231 conn(0x7fda8c0fd530 msgr2=0x7fda8c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.906+0000 7fda90c20640 1 -- 172.21.15.132:0/2094959231 shutdown_connections 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.906+0000 7fda90c20640 1 -- 172.21.15.132:0/2094959231 wait complete. 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda90c20640 1 Processor -- start 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda90c20640 1 -- start start 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda90c20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda8c101fa0 0x7fda8c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda90c20640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c103190 0x7fda8c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda90c20640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda8c103b50 0x7fda8c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda90c20640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c10dae0 con 0x7fda8c103190 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda90c20640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c10d960 con 0x7fda8c103b50 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda90c20640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c10dc60 con 0x7fda8c101fa0 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda89d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c103190 0x7fda8c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda89d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c103190 0x7fda8c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43840/0 (socket says 172.21.15.132:43840) 2024-04-23T18:33:56.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda89d9b640 1 -- 172.21.15.132:0/411419458 learned_addr learned my addr 172.21.15.132:0/411419458 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda8a59c640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda8c101fa0 0x7fda8c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda8ad9d640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda8c103b50 0x7fda8c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda89d9b640 1 -- 172.21.15.132:0/411419458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda8c101fa0 msgr2=0x7fda8c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda89d9b640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda8c101fa0 0x7fda8c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda89d9b640 1 -- 172.21.15.132:0/411419458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda8c103b50 msgr2=0x7fda8c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda89d9b640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda8c103b50 0x7fda8c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda89d9b640 1 -- 172.21.15.132:0/411419458 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c19fdf0 con 0x7fda8c103190 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.907+0000 7fda8ad9d640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda8c103b50 0x7fda8c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.908+0000 7fda89d9b640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c103190 0x7fda8c198710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda80005940 tx=0x7fda80031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.908+0000 7fda7b7fe640 1 -- 172.21.15.132:0/411419458 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8003f030 con 0x7fda8c103190 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.908+0000 7fda7b7fe640 1 -- 172.21.15.132:0/411419458 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda80003aa0 con 0x7fda8c103190 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.908+0000 7fda90c20640 1 -- 172.21.15.132:0/411419458 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8c1a0080 con 0x7fda8c103190 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.908+0000 7fda7b7fe640 1 -- 172.21.15.132:0/411419458 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80035070 con 0x7fda8c103190 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.908+0000 7fda90c20640 1 -- 172.21.15.132:0/411419458 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8c1a0570 con 0x7fda8c103190 2024-04-23T18:33:56.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.909+0000 7fda90c20640 1 -- 172.21.15.132:0/411419458 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8c109310 con 0x7fda8c103190 2024-04-23T18:33:56.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.910+0000 7fda7b7fe640 1 -- 172.21.15.132:0/411419458 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fda80042030 con 0x7fda8c103190 2024-04-23T18:33:56.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.911+0000 7fda7b7fe640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fda740a3bb0 0x7fda740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:56.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.911+0000 7fda8a59c640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fda740a3bb0 0x7fda740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:56.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.911+0000 7fda7b7fe640 1 -- 172.21.15.132:0/411419458 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fda800f05a0 con 0x7fda8c103190 2024-04-23T18:33:56.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.911+0000 7fda8a59c640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fda740a3bb0 0x7fda740a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fda8c102ff0 tx=0x7fda7c009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:56.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:56.916+0000 7fda7b7fe640 1 -- 172.21.15.132:0/411419458 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda800bd420 con 0x7fda8c103190 2024-04-23T18:33:57.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.134+0000 7fda90c20640 1 -- 172.21.15.132:0/411419458 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda8c02d810 con 0x7fda8c103190 2024-04-23T18:33:57.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.136+0000 7fda7b7fe640 1 -- 172.21.15.132:0/411419458 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10497 (secure 0 0 0) 0x7fda800bcd20 con 0x7fda8c103190 2024-04-23T18:33:57.128 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:57.129 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-23T18:33:57.129 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 13 2024-04-23T18:33:57.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.138+0000 7fda90c20640 1 -- 172.21.15.132:0/411419458 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fda740a3bb0 msgr2=0x7fda740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.138+0000 7fda90c20640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fda740a3bb0 0x7fda740a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fda8c102ff0 tx=0x7fda7c009720 comp rx=0 tx=0).stop 2024-04-23T18:33:57.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.138+0000 7fda90c20640 1 -- 172.21.15.132:0/411419458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c103190 msgr2=0x7fda8c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.138+0000 7fda90c20640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c103190 0x7fda8c198710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda80005940 tx=0x7fda80031a50 comp rx=0 tx=0).stop 2024-04-23T18:33:57.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.138+0000 7fda90c20640 1 -- 172.21.15.132:0/411419458 shutdown_connections 2024-04-23T18:33:57.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.138+0000 7fda90c20640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fda740a3bb0 0x7fda740a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.138+0000 7fda90c20640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda8c103b50 0x7fda8c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.138+0000 7fda90c20640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c103190 0x7fda8c198710 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.138+0000 7fda90c20640 1 --2- 172.21.15.132:0/411419458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda8c101fa0 0x7fda8c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.138+0000 7fda90c20640 1 -- 172.21.15.132:0/411419458 >> 172.21.15.132:0/411419458 conn(0x7fda8c0fd530 msgr2=0x7fda8c1064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:57.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.138+0000 7fda90c20640 1 -- 172.21.15.132:0/411419458 shutdown_connections 2024-04-23T18:33:57.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.138+0000 7fda90c20640 1 -- 172.21.15.132:0/411419458 wait complete. 2024-04-23T18:33:57.143 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:33:57.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.251+0000 7f888a60b640 1 -- 172.21.15.132:0/36866177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8884101fe0 msgr2=0x7f8884102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.251+0000 7f888a60b640 1 --2- 172.21.15.132:0/36866177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8884101fe0 0x7f8884102460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8874009900 tx=0x7f887402a410 comp rx=0 tx=0).stop 2024-04-23T18:33:57.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.251+0000 7f888a60b640 1 -- 172.21.15.132:0/36866177 shutdown_connections 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.251+0000 7f888a60b640 1 --2- 172.21.15.132:0/36866177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88841029a0 0x7f88841092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.251+0000 7f888a60b640 1 --2- 172.21.15.132:0/36866177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8884101fe0 0x7f8884102460 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.251+0000 7f888a60b640 1 --2- 172.21.15.132:0/36866177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8884100e90 0x7f88841012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.251+0000 7f888a60b640 1 -- 172.21.15.132:0/36866177 >> 172.21.15.132:0/36866177 conn(0x7f88840fc400 msgr2=0x7f88840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.252+0000 7f888a60b640 1 -- 172.21.15.132:0/36866177 shutdown_connections 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.252+0000 7f888a60b640 1 -- 172.21.15.132:0/36866177 wait complete. 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.252+0000 7f888a60b640 1 Processor -- start 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.252+0000 7f888a60b640 1 -- start start 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.252+0000 7f888a60b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8884100e90 0x7f88841981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.252+0000 7f888a60b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8884101fe0 0x7f8884198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.252+0000 7f888a60b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88841029a0 0x7f888419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.252+0000 7f888a60b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f888410c9d0 con 0x7f88841029a0 2024-04-23T18:33:57.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.252+0000 7f888a60b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f888410c850 con 0x7f8884101fe0 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.252+0000 7f888a60b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f888410cb50 con 0x7f8884100e90 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.252+0000 7f8888ba8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88841029a0 0x7f888419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f8888ba8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88841029a0 0x7f888419f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43870/0 (socket says 172.21.15.132:43870) 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f8883fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8884100e90 0x7f88841981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f8888ba8640 1 -- 172.21.15.132:0/83694915 learned_addr learned my addr 172.21.15.132:0/83694915 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f88837fe640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8884101fe0 0x7f8884198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f8888ba8640 1 -- 172.21.15.132:0/83694915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8884100e90 msgr2=0x7f88841981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f8888ba8640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8884100e90 0x7f88841981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f8888ba8640 1 -- 172.21.15.132:0/83694915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8884101fe0 msgr2=0x7f8884198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f8888ba8640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8884101fe0 0x7f8884198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f8888ba8640 1 -- 172.21.15.132:0/83694915 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888419fe10 con 0x7f88841029a0 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f88837fe640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8884101fe0 0x7f8884198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:57.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f8888ba8640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88841029a0 0x7f888419f7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f887c009fd0 tx=0x7f887c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:57.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f88817fa640 1 -- 172.21.15.132:0/83694915 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c017070 con 0x7f88841029a0 2024-04-23T18:33:57.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f88817fa640 1 -- 172.21.15.132:0/83694915 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f887c003e20 con 0x7f88841029a0 2024-04-23T18:33:57.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f888a60b640 1 -- 172.21.15.132:0/83694915 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88841a0100 con 0x7f88841029a0 2024-04-23T18:33:57.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f88817fa640 1 -- 172.21.15.132:0/83694915 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c010460 con 0x7f88841029a0 2024-04-23T18:33:57.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.253+0000 7f888a60b640 1 -- 172.21.15.132:0/83694915 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88841a05d0 con 0x7f88841029a0 2024-04-23T18:33:57.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.254+0000 7f888a60b640 1 -- 172.21.15.132:0/83694915 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88841a0830 con 0x7f88841029a0 2024-04-23T18:33:57.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.255+0000 7f88817fa640 1 -- 172.21.15.132:0/83694915 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f887c003970 con 0x7f88841029a0 2024-04-23T18:33:57.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.256+0000 7f88817fa640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f886c0a3bb0 0x7f886c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:57.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.256+0000 7f8883fff640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f886c0a3bb0 0x7f886c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:57.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.256+0000 7f88817fa640 1 -- 172.21.15.132:0/83694915 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f887c0cfd10 con 0x7f88841029a0 2024-04-23T18:33:57.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.256+0000 7f8883fff640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f886c0a3bb0 0x7f886c0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f887800bbe0 tx=0x7f887800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:57.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.260+0000 7f88817fa640 1 -- 172.21.15.132:0/83694915 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f887c09cb90 con 0x7f88841029a0 2024-04-23T18:33:57.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.448+0000 7f888a60b640 1 -- 172.21.15.132:0/83694915 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88841a0eb0 con 0x7f88841029a0 2024-04-23T18:33:57.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.449+0000 7f88817fa640 1 -- 172.21.15.132:0/83694915 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18538 (secure 0 0 0) 0x7f887c09c490 con 0x7f88841029a0 2024-04-23T18:33:57.442 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:57.442 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":82,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:33:11.819374+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:33:57.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.451+0000 7f888a60b640 1 -- 172.21.15.132:0/83694915 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f886c0a3bb0 msgr2=0x7f886c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.451+0000 7f888a60b640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f886c0a3bb0 0x7f886c0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f887800bbe0 tx=0x7f887800b390 comp rx=0 tx=0).stop 2024-04-23T18:33:57.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.451+0000 7f888a60b640 1 -- 172.21.15.132:0/83694915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88841029a0 msgr2=0x7f888419f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.451+0000 7f888a60b640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88841029a0 0x7f888419f7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f887c009fd0 tx=0x7f887c00b260 comp rx=0 tx=0).stop 2024-04-23T18:33:57.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.452+0000 7f888a60b640 1 -- 172.21.15.132:0/83694915 shutdown_connections 2024-04-23T18:33:57.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.452+0000 7f888a60b640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f886c0a3bb0 0x7f886c0a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.452+0000 7f888a60b640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88841029a0 0x7f888419f7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.452+0000 7f888a60b640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8884101fe0 0x7f8884198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.452+0000 7f888a60b640 1 --2- 172.21.15.132:0/83694915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8884100e90 0x7f88841981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.452+0000 7f888a60b640 1 -- 172.21.15.132:0/83694915 >> 172.21.15.132:0/83694915 conn(0x7f88840fc400 msgr2=0x7f8884069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:57.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.452+0000 7f888a60b640 1 -- 172.21.15.132:0/83694915 shutdown_connections 2024-04-23T18:33:57.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.452+0000 7f888a60b640 1 -- 172.21.15.132:0/83694915 wait complete. 2024-04-23T18:33:57.454 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:57 smithi132 ceph-mon[29178]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:57.454 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:57.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:57.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:57 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:57 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/411419458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:57.456 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:29:20.132451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:33:57.457 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:33:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:57 smithi148 ceph-mon[32311]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:57.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:57 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:57 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/411419458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:57.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:57 smithi157 ceph-mon[32580]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:57.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:57 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:57.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:57 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/411419458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:57.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.569+0000 7f4570a7c640 1 -- 172.21.15.132:0/1984160702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f456c103b50 msgr2=0x7f456c10a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.569+0000 7f4570a7c640 1 --2- 172.21.15.132:0/1984160702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f456c103b50 0x7f456c10a3e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4564009ca0 tx=0x7f456402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.569+0000 7f4570a7c640 1 -- 172.21.15.132:0/1984160702 shutdown_connections 2024-04-23T18:33:57.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.569+0000 7f4570a7c640 1 --2- 172.21.15.132:0/1984160702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f456c103b50 0x7f456c10a3e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.569+0000 7f4570a7c640 1 --2- 172.21.15.132:0/1984160702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c103190 0x7f456c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.569+0000 7f4570a7c640 1 --2- 172.21.15.132:0/1984160702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c101fa0 0x7f456c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.569+0000 7f4570a7c640 1 -- 172.21.15.132:0/1984160702 >> 172.21.15.132:0/1984160702 conn(0x7f456c0fd530 msgr2=0x7f456c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:57.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.569+0000 7f4570a7c640 1 -- 172.21.15.132:0/1984160702 shutdown_connections 2024-04-23T18:33:57.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.569+0000 7f4570a7c640 1 -- 172.21.15.132:0/1984160702 wait complete. 2024-04-23T18:33:57.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f4570a7c640 1 Processor -- start 2024-04-23T18:33:57.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f4570a7c640 1 -- start start 2024-04-23T18:33:57.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f4570a7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c101fa0 0x7f456c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:57.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f4570a7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c103190 0x7f456c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f4570a7c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f456c103b50 0x7f456c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f4570a7c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f456c10dae0 con 0x7f456c103b50 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f4570a7c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f456c10d960 con 0x7f456c103190 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f4570a7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f456c10dc60 con 0x7f456c101fa0 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f456ad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f456c103b50 0x7f456c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f456ad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f456c103b50 0x7f456c19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43890/0 (socket says 172.21.15.132:43890) 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f456ad9d640 1 -- 172.21.15.132:0/2694270237 learned_addr learned my addr 172.21.15.132:0/2694270237 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f456a59c640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c101fa0 0x7f456c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.570+0000 7f4569d9b640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c103190 0x7f456c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.571+0000 7f456a59c640 1 -- 172.21.15.132:0/2694270237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c103190 msgr2=0x7f456c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.571+0000 7f456a59c640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c103190 0x7f456c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.571+0000 7f456a59c640 1 -- 172.21.15.132:0/2694270237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f456c103b50 msgr2=0x7f456c19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.571+0000 7f456a59c640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f456c103b50 0x7f456c19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.571+0000 7f456a59c640 1 -- 172.21.15.132:0/2694270237 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c19fd80 con 0x7f456c101fa0 2024-04-23T18:33:57.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.571+0000 7f456ad9d640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f456c103b50 0x7f456c19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:57.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.571+0000 7f456a59c640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c101fa0 0x7f456c198160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f455c00e9c0 tx=0x7f455c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:57.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.571+0000 7f455b7fe640 1 -- 172.21.15.132:0/2694270237 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c00ccc0 con 0x7f456c101fa0 2024-04-23T18:33:57.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.571+0000 7f4570a7c640 1 -- 172.21.15.132:0/2694270237 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c1a0070 con 0x7f456c101fa0 2024-04-23T18:33:57.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.571+0000 7f455b7fe640 1 -- 172.21.15.132:0/2694270237 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f455c0035f0 con 0x7f456c101fa0 2024-04-23T18:33:57.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.571+0000 7f455b7fe640 1 -- 172.21.15.132:0/2694270237 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c00ce60 con 0x7f456c101fa0 2024-04-23T18:33:57.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.571+0000 7f4570a7c640 1 -- 172.21.15.132:0/2694270237 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c1a05c0 con 0x7f456c101fa0 2024-04-23T18:33:57.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.572+0000 7f4570a7c640 1 -- 172.21.15.132:0/2694270237 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456c1092c0 con 0x7f456c101fa0 2024-04-23T18:33:57.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.573+0000 7f455b7fe640 1 -- 172.21.15.132:0/2694270237 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f455c004000 con 0x7f456c101fa0 2024-04-23T18:33:57.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.573+0000 7f455b7fe640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45540a3bb0 0x7f45540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:57.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.574+0000 7f4569d9b640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45540a3bb0 0x7f45540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:57.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.574+0000 7f455b7fe640 1 -- 172.21.15.132:0/2694270237 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f455c0d02f0 con 0x7f456c101fa0 2024-04-23T18:33:57.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.574+0000 7f4569d9b640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45540a3bb0 0x7f45540a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f456c199670 tx=0x7f4560005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:57.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.577+0000 7f455b7fe640 1 -- 172.21.15.132:0/2694270237 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f455c09d110 con 0x7f456c101fa0 2024-04-23T18:33:57.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.764+0000 7f4570a7c640 1 -- 172.21.15.132:0/2694270237 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f456c02d870 con 0x7f456c101fa0 2024-04-23T18:33:57.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.765+0000 7f455b7fe640 1 -- 172.21.15.132:0/2694270237 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f455c09ca10 con 0x7f456c101fa0 2024-04-23T18:33:57.757 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-04-23T18:33:57.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.768+0000 7f4570a7c640 1 -- 172.21.15.132:0/2694270237 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45540a3bb0 msgr2=0x7f45540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.768+0000 7f4570a7c640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45540a3bb0 0x7f45540a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f456c199670 tx=0x7f4560005f90 comp rx=0 tx=0).stop 2024-04-23T18:33:57.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.768+0000 7f4570a7c640 1 -- 172.21.15.132:0/2694270237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c101fa0 msgr2=0x7f456c198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.768+0000 7f4570a7c640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c101fa0 0x7f456c198160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f455c00e9c0 tx=0x7f455c00ee80 comp rx=0 tx=0).stop 2024-04-23T18:33:57.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.768+0000 7f4570a7c640 1 -- 172.21.15.132:0/2694270237 shutdown_connections 2024-04-23T18:33:57.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.768+0000 7f4570a7c640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45540a3bb0 0x7f45540a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.768+0000 7f4570a7c640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f456c103b50 0x7f456c19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.768+0000 7f4570a7c640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c103190 0x7f456c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.768+0000 7f4570a7c640 1 --2- 172.21.15.132:0/2694270237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c101fa0 0x7f456c198160 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.768+0000 7f4570a7c640 1 -- 172.21.15.132:0/2694270237 >> 172.21.15.132:0/2694270237 conn(0x7f456c0fd530 msgr2=0x7f456c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:57.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.768+0000 7f4570a7c640 1 -- 172.21.15.132:0/2694270237 shutdown_connections 2024-04-23T18:33:57.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.768+0000 7f4570a7c640 1 -- 172.21.15.132:0/2694270237 wait complete. 2024-04-23T18:33:57.775 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T18:33:57.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.895+0000 7f56bca6b640 1 -- 172.21.15.132:0/1754826745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56b8103c00 msgr2=0x7f56b810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.895+0000 7f56bca6b640 1 --2- 172.21.15.132:0/1754826745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56b8103c00 0x7f56b810a490 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f56b0009cd0 tx=0x7f56b002a8e0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.895+0000 7f56bca6b640 1 -- 172.21.15.132:0/1754826745 shutdown_connections 2024-04-23T18:33:57.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.895+0000 7f56bca6b640 1 --2- 172.21.15.132:0/1754826745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b8103240 0x7f56b81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.895+0000 7f56bca6b640 1 --2- 172.21.15.132:0/1754826745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56b8103c00 0x7f56b810a490 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.895+0000 7f56bca6b640 1 --2- 172.21.15.132:0/1754826745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b8100270 0x7f56b8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.895+0000 7f56bca6b640 1 -- 172.21.15.132:0/1754826745 >> 172.21.15.132:0/1754826745 conn(0x7f56b80fbbc0 msgr2=0x7f56b80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:57.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.895+0000 7f56bca6b640 1 -- 172.21.15.132:0/1754826745 shutdown_connections 2024-04-23T18:33:57.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.895+0000 7f56bca6b640 1 -- 172.21.15.132:0/1754826745 wait complete. 2024-04-23T18:33:57.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.896+0000 7f56bca6b640 1 Processor -- start 2024-04-23T18:33:57.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.896+0000 7f56bca6b640 1 -- start start 2024-04-23T18:33:57.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.896+0000 7f56bca6b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b8100270 0x7f56b8198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:57.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.896+0000 7f56bca6b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56b8103240 0x7f56b8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:57.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.896+0000 7f56bca6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b8103c00 0x7f56b819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:57.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.896+0000 7f56bca6b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f56b810db50 con 0x7f56b8103240 2024-04-23T18:33:57.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.896+0000 7f56bca6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56b810d9d0 con 0x7f56b8103c00 2024-04-23T18:33:57.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.896+0000 7f56bca6b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56b810dcd0 con 0x7f56b8100270 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.896+0000 7f56b659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b8100270 0x7f56b8198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56b659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b8100270 0x7f56b8198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.132:40902/0 (socket says 172.21.15.132:40902) 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56b659c640 1 -- 172.21.15.132:0/3164772223 learned_addr learned my addr 172.21.15.132:0/3164772223 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56affff640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56b8103240 0x7f56b8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56b6d9d640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b8103c00 0x7f56b819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56b659c640 1 -- 172.21.15.132:0/3164772223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b8103c00 msgr2=0x7f56b819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56b659c640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b8103c00 0x7f56b819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56b659c640 1 -- 172.21.15.132:0/3164772223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56b8103240 msgr2=0x7f56b8198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56b659c640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56b8103240 0x7f56b8198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56b659c640 1 -- 172.21.15.132:0/3164772223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b81010b0 con 0x7f56b8100270 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56b6d9d640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b8103c00 0x7f56b819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56b659c640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b8100270 0x7f56b8198220 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f56a800d950 tx=0x7f56a800dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:57.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56af7fe640 1 -- 172.21.15.132:0/3164772223 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8017020 con 0x7f56b8100270 2024-04-23T18:33:57.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56bca6b640 1 -- 172.21.15.132:0/3164772223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b81013a0 con 0x7f56b8100270 2024-04-23T18:33:57.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56af7fe640 1 -- 172.21.15.132:0/3164772223 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a8009c30 con 0x7f56b8100270 2024-04-23T18:33:57.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56af7fe640 1 -- 172.21.15.132:0/3164772223 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a800f620 con 0x7f56b8100270 2024-04-23T18:33:57.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.897+0000 7f56bca6b640 1 -- 172.21.15.132:0/3164772223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b81018f0 con 0x7f56b8100270 2024-04-23T18:33:57.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.898+0000 7f56bca6b640 1 -- 172.21.15.132:0/3164772223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b8102d30 con 0x7f56b8100270 2024-04-23T18:33:57.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.899+0000 7f56af7fe640 1 -- 172.21.15.132:0/3164772223 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f56a801a020 con 0x7f56b8100270 2024-04-23T18:33:57.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.900+0000 7f56af7fe640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f56a40a3bb0 0x7f56a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:57.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.900+0000 7f56affff640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f56a40a3bb0 0x7f56a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:57.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.900+0000 7f56af7fe640 1 -- 172.21.15.132:0/3164772223 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f56a80cfac0 con 0x7f56b8100270 2024-04-23T18:33:57.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.900+0000 7f56affff640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f56a40a3bb0 0x7f56a40a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f56b8199730 tx=0x7f56a0009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:57.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:57.904+0000 7f56af7fe640 1 -- 172.21.15.132:0/3164772223 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56a8021070 con 0x7f56b8100270 2024-04-23T18:33:58.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.085+0000 7f56bca6b640 1 -- 172.21.15.132:0/3164772223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f56b8109100 con 0x7f56b8100270 2024-04-23T18:33:58.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.085+0000 7f56af7fe640 1 -- 172.21.15.132:0/3164772223 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f56a8009de0 con 0x7f56b8100270 2024-04-23T18:33:58.078 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:33:58.078 INFO:teuthology.orchestra.run.smithi132.stdout:[ 2024-04-23T18:33:58.078 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:33:58.078 INFO:teuthology.orchestra.run.smithi132.stdout: "poolnum": 1, 2024-04-23T18:33:58.078 INFO:teuthology.orchestra.run.smithi132.stdout: "poolname": ".mgr" 2024-04-23T18:33:58.078 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:33:58.078 INFO:teuthology.orchestra.run.smithi132.stdout:] 2024-04-23T18:33:58.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.087+0000 7f56bca6b640 1 -- 172.21.15.132:0/3164772223 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f56a40a3bb0 msgr2=0x7f56a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:58.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.087+0000 7f56bca6b640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f56a40a3bb0 0x7f56a40a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f56b8199730 tx=0x7f56a0009110 comp rx=0 tx=0).stop 2024-04-23T18:33:58.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.087+0000 7f56bca6b640 1 -- 172.21.15.132:0/3164772223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b8100270 msgr2=0x7f56b8198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:58.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.087+0000 7f56bca6b640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b8100270 0x7f56b8198220 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f56a800d950 tx=0x7f56a800dd50 comp rx=0 tx=0).stop 2024-04-23T18:33:58.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.087+0000 7f56bca6b640 1 -- 172.21.15.132:0/3164772223 shutdown_connections 2024-04-23T18:33:58.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.087+0000 7f56bca6b640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f56a40a3bb0 0x7f56a40a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:58.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.087+0000 7f56bca6b640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b8103c00 0x7f56b819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:58.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.087+0000 7f56bca6b640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56b8103240 0x7f56b8198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:58.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.087+0000 7f56bca6b640 1 --2- 172.21.15.132:0/3164772223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56b8100270 0x7f56b8198220 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:58.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.087+0000 7f56bca6b640 1 -- 172.21.15.132:0/3164772223 >> 172.21.15.132:0/3164772223 conn(0x7f56b80fbbc0 msgr2=0x7f56b81065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:58.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.087+0000 7f56bca6b640 1 -- 172.21.15.132:0/3164772223 shutdown_connections 2024-04-23T18:33:58.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.087+0000 7f56bca6b640 1 -- 172.21.15.132:0/3164772223 wait complete. 2024-04-23T18:33:58.091 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T18:33:58.091 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T18:33:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.207+0000 7fc154fa2640 1 -- 172.21.15.132:0/4186973072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1501053e0 msgr2=0x7fc1501077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.207+0000 7fc154fa2640 1 --2- 172.21.15.132:0/4186973072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1501053e0 0x7fc1501077d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc144009900 tx=0x7fc14402a410 comp rx=0 tx=0).stop 2024-04-23T18:33:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.207+0000 7fc154fa2640 1 -- 172.21.15.132:0/4186973072 shutdown_connections 2024-04-23T18:33:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.207+0000 7fc154fa2640 1 --2- 172.21.15.132:0/4186973072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc150107d10 0x7fc15010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.207+0000 7fc154fa2640 1 --2- 172.21.15.132:0/4186973072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1501053e0 0x7fc1501077d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.207+0000 7fc154fa2640 1 --2- 172.21.15.132:0/4186973072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc150102ab0 0x7fc150104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.207+0000 7fc154fa2640 1 -- 172.21.15.132:0/4186973072 >> 172.21.15.132:0/4186973072 conn(0x7fc1500fc440 msgr2=0x7fc1500fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.207+0000 7fc154fa2640 1 -- 172.21.15.132:0/4186973072 shutdown_connections 2024-04-23T18:33:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.207+0000 7fc154fa2640 1 -- 172.21.15.132:0/4186973072 wait complete. 2024-04-23T18:33:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc154fa2640 1 Processor -- start 2024-04-23T18:33:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc154fa2640 1 -- start start 2024-04-23T18:33:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc154fa2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc150102ab0 0x7fc150198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc154fa2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1501053e0 0x7fc150198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc154fa2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc150107d10 0x7fc15019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc154fa2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc15010daf0 con 0x7fc150102ab0 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc154fa2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc15010d970 con 0x7fc1501053e0 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc154fa2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc15010dc70 con 0x7fc150107d10 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc14e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc150102ab0 0x7fc150198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc14e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc150102ab0 0x7fc150198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43932/0 (socket says 172.21.15.132:43932) 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc14e59c640 1 -- 172.21.15.132:0/1720672999 learned_addr learned my addr 172.21.15.132:0/1720672999 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc14ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc150107d10 0x7fc15019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.208+0000 7fc14dd9b640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1501053e0 0x7fc150198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc14ed9d640 1 -- 172.21.15.132:0/1720672999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1501053e0 msgr2=0x7fc150198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc14ed9d640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1501053e0 0x7fc150198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc14ed9d640 1 -- 172.21.15.132:0/1720672999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc150102ab0 msgr2=0x7fc150198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc14ed9d640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc150102ab0 0x7fc150198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc14ed9d640 1 -- 172.21.15.132:0/1720672999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15019fe50 con 0x7fc150107d10 2024-04-23T18:33:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc14e59c640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc150102ab0 0x7fc150198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc14dd9b640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1501053e0 0x7fc150198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc14ed9d640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc150107d10 0x7fc15019f7e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc148009fd0 tx=0x7fc14800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc13f7fe640 1 -- 172.21.15.132:0/1720672999 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14800ce20 con 0x7fc150107d10 2024-04-23T18:33:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc13f7fe640 1 -- 172.21.15.132:0/1720672999 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc148005020 con 0x7fc150107d10 2024-04-23T18:33:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc154fa2640 1 -- 172.21.15.132:0/1720672999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1501a0140 con 0x7fc150107d10 2024-04-23T18:33:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc13f7fe640 1 -- 172.21.15.132:0/1720672999 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148010690 con 0x7fc150107d10 2024-04-23T18:33:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.209+0000 7fc154fa2640 1 -- 172.21.15.132:0/1720672999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1501a0610 con 0x7fc150107d10 2024-04-23T18:33:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.210+0000 7fc154fa2640 1 -- 172.21.15.132:0/1720672999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1500fdf30 con 0x7fc150107d10 2024-04-23T18:33:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.211+0000 7fc13f7fe640 1 -- 172.21.15.132:0/1720672999 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc1480054d0 con 0x7fc150107d10 2024-04-23T18:33:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.212+0000 7fc13f7fe640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc1380a3bb0 0x7fc1380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.212+0000 7fc14e59c640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc1380a3bb0 0x7fc1380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.212+0000 7fc13f7fe640 1 -- 172.21.15.132:0/1720672999 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc1480d0bb0 con 0x7fc150107d10 2024-04-23T18:33:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.212+0000 7fc14e59c640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc1380a3bb0 0x7fc1380a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc140011550 tx=0x7fc140005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.215+0000 7fc13f7fe640 1 -- 172.21.15.132:0/1720672999 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc14809d990 con 0x7fc150107d10 2024-04-23T18:33:58.345 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:58 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:58.345 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:58 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/83694915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:33:58.345 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:58 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2694270237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:33:58.345 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:58 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3164772223' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T18:33:58.394 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:58.402+0000 7fc154fa2640 1 -- 172.21.15.132:0/1720672999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc15002d5c0 con 0x7fc150107d10 2024-04-23T18:33:58.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:58 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:58.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:58 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/83694915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:33:58.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:58 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2694270237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:33:58.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:58 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3164772223' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T18:33:58.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:58 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:58.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:58 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/83694915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:33:58.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:58 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2694270237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:33:58.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:58 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3164772223' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T18:33:59.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.204+0000 7fc13f7fe640 1 -- 172.21.15.132:0/1720672999 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fc14809d290 con 0x7fc150107d10 2024-04-23T18:33:59.197 INFO:teuthology.orchestra.run.smithi132.stderr:pool 'cephfs_metadata' created 2024-04-23T18:33:59.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.208+0000 7fc154fa2640 1 -- 172.21.15.132:0/1720672999 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc1380a3bb0 msgr2=0x7fc1380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:59.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.208+0000 7fc154fa2640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc1380a3bb0 0x7fc1380a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc140011550 tx=0x7fc140005f90 comp rx=0 tx=0).stop 2024-04-23T18:33:59.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.208+0000 7fc154fa2640 1 -- 172.21.15.132:0/1720672999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc150107d10 msgr2=0x7fc15019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:59.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.208+0000 7fc154fa2640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc150107d10 0x7fc15019f7e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc148009fd0 tx=0x7fc14800ef10 comp rx=0 tx=0).stop 2024-04-23T18:33:59.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.209+0000 7fc154fa2640 1 -- 172.21.15.132:0/1720672999 shutdown_connections 2024-04-23T18:33:59.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.209+0000 7fc154fa2640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc1380a3bb0 0x7fc1380a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:59.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.209+0000 7fc154fa2640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc150107d10 0x7fc15019f7e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:59.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.209+0000 7fc154fa2640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1501053e0 0x7fc150198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:59.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.209+0000 7fc154fa2640 1 --2- 172.21.15.132:0/1720672999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc150102ab0 0x7fc150198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:59.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.209+0000 7fc154fa2640 1 -- 172.21.15.132:0/1720672999 >> 172.21.15.132:0/1720672999 conn(0x7fc1500fc440 msgr2=0x7fc150104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:59.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.209+0000 7fc154fa2640 1 -- 172.21.15.132:0/1720672999 shutdown_connections 2024-04-23T18:33:59.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.209+0000 7fc154fa2640 1 -- 172.21.15.132:0/1720672999 wait complete. 2024-04-23T18:33:59.213 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T18:33:59.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 -- 172.21.15.132:0/112255935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32181029a0 msgr2=0x7f32181092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:59.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 --2- 172.21.15.132:0/112255935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32181029a0 0x7f32181092c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3210009ca0 tx=0x7f321002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:33:59.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 -- 172.21.15.132:0/112255935 shutdown_connections 2024-04-23T18:33:59.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 --2- 172.21.15.132:0/112255935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32181029a0 0x7f32181092c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 --2- 172.21.15.132:0/112255935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3218101fe0 0x7f3218102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 --2- 172.21.15.132:0/112255935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3218100e90 0x7f32181012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 -- 172.21.15.132:0/112255935 >> 172.21.15.132:0/112255935 conn(0x7f32180fc440 msgr2=0x7f32180fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 -- 172.21.15.132:0/112255935 shutdown_connections 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 -- 172.21.15.132:0/112255935 wait complete. 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 Processor -- start 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 -- start start 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3218100e90 0x7f3218198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3218101fe0 0x7f3218198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32181029a0 0x7f321819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f321810c9f0 con 0x7f3218101fe0 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f321810c870 con 0x7f3218100e90 2024-04-23T18:33:59.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.327+0000 7f321fae1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f321810cb70 con 0x7f32181029a0 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321d07c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3218101fe0 0x7f3218198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321d07c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3218101fe0 0x7f3218198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43948/0 (socket says 172.21.15.132:43948) 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321d07c640 1 -- 172.21.15.132:0/334906725 learned_addr learned my addr 172.21.15.132:0/334906725 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321d87d640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3218100e90 0x7f3218198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321e07e640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32181029a0 0x7f321819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321d07c640 1 -- 172.21.15.132:0/334906725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32181029a0 msgr2=0x7f321819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321d07c640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32181029a0 0x7f321819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321d07c640 1 -- 172.21.15.132:0/334906725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3218100e90 msgr2=0x7f3218198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321d07c640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3218100e90 0x7f3218198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321d07c640 1 -- 172.21.15.132:0/334906725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321819fd60 con 0x7f3218101fe0 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321d07c640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3218101fe0 0x7f3218198680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f320800c970 tx=0x7f320800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f320effd640 1 -- 172.21.15.132:0/334906725 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208004220 con 0x7f3218101fe0 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321fae1640 1 -- 172.21.15.132:0/334906725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321819fff0 con 0x7f3218101fe0 2024-04-23T18:33:59.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f320effd640 1 -- 172.21.15.132:0/334906725 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32080043c0 con 0x7f3218101fe0 2024-04-23T18:33:59.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f320effd640 1 -- 172.21.15.132:0/334906725 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320800f450 con 0x7f3218101fe0 2024-04-23T18:33:59.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.328+0000 7f321fae1640 1 -- 172.21.15.132:0/334906725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32181a0560 con 0x7f3218101fe0 2024-04-23T18:33:59.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.329+0000 7f321fae1640 1 -- 172.21.15.132:0/334906725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32181a07f0 con 0x7f3218101fe0 2024-04-23T18:33:59.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.330+0000 7f320effd640 1 -- 172.21.15.132:0/334906725 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f3208003610 con 0x7f3218101fe0 2024-04-23T18:33:59.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.331+0000 7f320effd640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f32040a3bb0 0x7f32040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:59.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.331+0000 7f321d87d640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f32040a3bb0 0x7f32040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:59.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.331+0000 7f320effd640 1 -- 172.21.15.132:0/334906725 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f32080cfe40 con 0x7f3218101fe0 2024-04-23T18:33:59.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.331+0000 7f321d87d640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f32040a3bb0 0x7f32040a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3214005fd0 tx=0x7f321400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:59.327 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.336+0000 7f320effd640 1 -- 172.21.15.132:0/334906725 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3208014030 con 0x7f3218101fe0 2024-04-23T18:33:59.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:33:59.513+0000 7f321fae1640 1 -- 172.21.15.132:0/334906725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f32181a09c0 con 0x7f3218101fe0 2024-04-23T18:33:59.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:59.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1720672999' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:33:59.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:33:59.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:59.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:59.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:33:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1720672999' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:33:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:33:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:59.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:59.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.541 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:33:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:33:59.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1720672999' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:33:59.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:33:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:33:59.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:33:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:34:00.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.208+0000 7f320effd640 1 -- 172.21.15.132:0/334906725 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f320809c500 con 0x7f3218101fe0 2024-04-23T18:34:00.201 INFO:teuthology.orchestra.run.smithi132.stderr:pool 'cephfs_data' created 2024-04-23T18:34:00.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.212+0000 7f321fae1640 1 -- 172.21.15.132:0/334906725 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f32040a3bb0 msgr2=0x7f32040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:00.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.212+0000 7f321fae1640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f32040a3bb0 0x7f32040a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3214005fd0 tx=0x7f321400b390 comp rx=0 tx=0).stop 2024-04-23T18:34:00.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.212+0000 7f321fae1640 1 -- 172.21.15.132:0/334906725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3218101fe0 msgr2=0x7f3218198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:00.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.212+0000 7f321fae1640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3218101fe0 0x7f3218198680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f320800c970 tx=0x7f320800ce30 comp rx=0 tx=0).stop 2024-04-23T18:34:00.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.212+0000 7f321fae1640 1 -- 172.21.15.132:0/334906725 shutdown_connections 2024-04-23T18:34:00.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.212+0000 7f321fae1640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f32040a3bb0 0x7f32040a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:00.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.212+0000 7f321fae1640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32181029a0 0x7f321819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:00.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.212+0000 7f321fae1640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3218101fe0 0x7f3218198680 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:00.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.212+0000 7f321fae1640 1 --2- 172.21.15.132:0/334906725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3218100e90 0x7f3218198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:00.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.212+0000 7f321fae1640 1 -- 172.21.15.132:0/334906725 >> 172.21.15.132:0/334906725 conn(0x7f32180fc440 msgr2=0x7f321806a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:00.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.213+0000 7f321fae1640 1 -- 172.21.15.132:0/334906725 shutdown_connections 2024-04-23T18:34:00.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.213+0000 7f321fae1640 1 -- 172.21.15.132:0/334906725 wait complete. 2024-04-23T18:34:00.231 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T18:34:00.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.344+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1564498303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cac102040 msgr2=0x7f2cac1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:00.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.344+0000 7f2cb46e1640 1 --2- 172.21.15.132:0/1564498303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cac102040 0x7f2cac1024c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009900 tx=0x7f2c9c02a410 comp rx=0 tx=0).stop 2024-04-23T18:34:00.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.344+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1564498303 shutdown_connections 2024-04-23T18:34:00.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.344+0000 7f2cb46e1640 1 --2- 172.21.15.132:0/1564498303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac102a00 0x7f2cac109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:00.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.344+0000 7f2cb46e1640 1 --2- 172.21.15.132:0/1564498303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cac102040 0x7f2cac1024c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:00.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.344+0000 7f2cb46e1640 1 --2- 172.21.15.132:0/1564498303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cac100e60 0x7f2cac101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:00.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.344+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1564498303 >> 172.21.15.132:0/1564498303 conn(0x7f2cac0fc440 msgr2=0x7f2cac0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:00.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.344+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1564498303 shutdown_connections 2024-04-23T18:34:00.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.344+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1564498303 wait complete. 2024-04-23T18:34:00.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb46e1640 1 Processor -- start 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb46e1640 1 -- start start 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb46e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac100e60 0x7f2cac06b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb46e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cac102040 0x7f2cac19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb46e1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cac102a00 0x7f2cac19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb46e1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10ca40 con 0x7f2cac102a00 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb46e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10c8c0 con 0x7f2cac100e60 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb46e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10cbc0 con 0x7f2cac102040 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb2c7e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cac102a00 0x7f2cac19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb2c7e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cac102a00 0x7f2cac19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35862/0 (socket says 172.21.15.132:35862) 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb2c7e640 1 -- 172.21.15.132:0/1575014349 learned_addr learned my addr 172.21.15.132:0/1575014349 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb247d640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac100e60 0x7f2cac06b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb2c7e640 1 -- 172.21.15.132:0/1575014349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cac102040 msgr2=0x7f2cac19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb2c7e640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cac102040 0x7f2cac19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:00.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb2c7e640 1 -- 172.21.15.132:0/1575014349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac100e60 msgr2=0x7f2cac06b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:00.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb2c7e640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac100e60 0x7f2cac06b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:00.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb2c7e640 1 -- 172.21.15.132:0/1575014349 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac19fdb0 con 0x7f2cac102a00 2024-04-23T18:34:00.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb247d640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac100e60 0x7f2cac06b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:00.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb2c7e640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cac102a00 0x7f2cac19f4f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400cce0 tx=0x7f2ca400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:00.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2ca37fe640 1 -- 172.21.15.132:0/1575014349 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400f450 con 0x7f2cac102a00 2024-04-23T18:34:00.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2ca37fe640 1 -- 172.21.15.132:0/1575014349 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca400faa0 con 0x7f2cac102a00 2024-04-23T18:34:00.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1575014349 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac1a00a0 con 0x7f2cac102a00 2024-04-23T18:34:00.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.345+0000 7f2ca37fe640 1 -- 172.21.15.132:0/1575014349 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4018b70 con 0x7f2cac102a00 2024-04-23T18:34:00.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.346+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1575014349 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac1a05f0 con 0x7f2cac102a00 2024-04-23T18:34:00.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.346+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1575014349 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac108200 con 0x7f2cac102a00 2024-04-23T18:34:00.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.348+0000 7f2ca37fe640 1 -- 172.21.15.132:0/1575014349 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2ca4018d10 con 0x7f2cac102a00 2024-04-23T18:34:00.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.348+0000 7f2ca37fe640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2c980a3c30 0x7f2c980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:00.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.348+0000 7f2cb247d640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2c980a3c30 0x7f2c980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:00.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.348+0000 7f2ca37fe640 1 -- 172.21.15.132:0/1575014349 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f2ca40d07a0 con 0x7f2cac102a00 2024-04-23T18:34:00.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.349+0000 7f2cb247d640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2c980a3c30 0x7f2c980a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2cac101bb0 tx=0x7f2ca800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:00.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.352+0000 7f2ca37fe640 1 -- 172.21.15.132:0/1575014349 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ca4014030 con 0x7f2cac102a00 2024-04-23T18:34:00.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:00 smithi148 ceph-mon[32311]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T18:34:00.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:00 smithi148 ceph-mon[32311]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T18:34:00.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:00 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/334906725' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T18:34:00.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:00 smithi157 ceph-mon[32580]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T18:34:00.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:00 smithi157 ceph-mon[32580]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T18:34:00.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:00 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/334906725' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T18:34:00.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:00.570+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1575014349 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2cac02d5c0 con 0x7f2cac102a00 2024-04-23T18:34:00.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:00 smithi132 ceph-mon[29178]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T18:34:00.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:00 smithi132 ceph-mon[29178]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T18:34:00.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:00 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/334906725' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T18:34:01.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.514+0000 7f2ca37fe640 1 -- 172.21.15.132:0/1575014349 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f2ca409d440 con 0x7f2cac102a00 2024-04-23T18:34:01.506 INFO:teuthology.orchestra.run.smithi132.stderr:new fs with metadata pool 2 and data pool 3 2024-04-23T18:34:01.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.517+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1575014349 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2c980a3c30 msgr2=0x7f2c980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.517+0000 7f2cb46e1640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2c980a3c30 0x7f2c980a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2cac101bb0 tx=0x7f2ca800a560 comp rx=0 tx=0).stop 2024-04-23T18:34:01.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.517+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1575014349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cac102a00 msgr2=0x7f2cac19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.517+0000 7f2cb46e1640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cac102a00 0x7f2cac19f4f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400cce0 tx=0x7f2ca400a620 comp rx=0 tx=0).stop 2024-04-23T18:34:01.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.517+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1575014349 shutdown_connections 2024-04-23T18:34:01.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.517+0000 7f2cb46e1640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2c980a3c30 0x7f2c980a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.517+0000 7f2cb46e1640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cac102040 0x7f2cac19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.517+0000 7f2cb46e1640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cac102a00 0x7f2cac19f4f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.517+0000 7f2cb46e1640 1 --2- 172.21.15.132:0/1575014349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac100e60 0x7f2cac06b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.517+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1575014349 >> 172.21.15.132:0/1575014349 conn(0x7f2cac0fc440 msgr2=0x7f2cac1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:01.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.517+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1575014349 shutdown_connections 2024-04-23T18:34:01.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.517+0000 7f2cb46e1640 1 -- 172.21.15.132:0/1575014349 wait complete. 2024-04-23T18:34:01.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:01 smithi148 ceph-mon[32311]: pgmap v286: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:34:01.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:01 smithi148 ceph-mon[32311]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T18:34:01.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:01 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/334906725' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T18:34:01.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:01 smithi148 ceph-mon[32311]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T18:34:01.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:01 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1575014349' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T18:34:01.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:01 smithi148 ceph-mon[32311]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T18:34:01.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:01 smithi132 ceph-mon[29178]: pgmap v286: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:34:01.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:01 smithi132 ceph-mon[29178]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T18:34:01.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:01 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/334906725' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T18:34:01.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:01 smithi132 ceph-mon[29178]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T18:34:01.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:01 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1575014349' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T18:34:01.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:01 smithi132 ceph-mon[29178]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T18:34:01.525 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:34:01.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:01 smithi157 ceph-mon[32580]: pgmap v286: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:34:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:01 smithi157 ceph-mon[32580]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T18:34:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:01 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/334906725' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T18:34:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:01 smithi157 ceph-mon[32580]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T18:34:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:01 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1575014349' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T18:34:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:01 smithi157 ceph-mon[32580]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T18:34:01.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.637+0000 7f2f694cf640 1 -- 172.21.15.132:0/3277255053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f64100ea0 msgr2=0x7f2f641012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.637+0000 7f2f694cf640 1 --2- 172.21.15.132:0/3277255053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f64100ea0 0x7f2f641012c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2f54009900 tx=0x7f2f5402a410 comp rx=0 tx=0).stop 2024-04-23T18:34:01.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.637+0000 7f2f694cf640 1 -- 172.21.15.132:0/3277255053 shutdown_connections 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.637+0000 7f2f694cf640 1 --2- 172.21.15.132:0/3277255053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f64101ff0 0x7f2f64102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.637+0000 7f2f694cf640 1 --2- 172.21.15.132:0/3277255053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f641029b0 0x7f2f641092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.637+0000 7f2f694cf640 1 --2- 172.21.15.132:0/3277255053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f64100ea0 0x7f2f641012c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.637+0000 7f2f694cf640 1 -- 172.21.15.132:0/3277255053 >> 172.21.15.132:0/3277255053 conn(0x7f2f640fc430 msgr2=0x7f2f640fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.637+0000 7f2f694cf640 1 -- 172.21.15.132:0/3277255053 shutdown_connections 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.637+0000 7f2f694cf640 1 -- 172.21.15.132:0/3277255053 wait complete. 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.637+0000 7f2f694cf640 1 Processor -- start 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f694cf640 1 -- start start 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f694cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f64100ea0 0x7f2f64197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f694cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f64101ff0 0x7f2f64197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f694cf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f641029b0 0x7f2f6419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f694cf640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6410c9d0 con 0x7f2f641029b0 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f694cf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6410c850 con 0x7f2f64101ff0 2024-04-23T18:34:01.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f694cf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6410cb50 con 0x7f2f64100ea0 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f637fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f641029b0 0x7f2f6419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f637fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f641029b0 0x7f2f6419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35890/0 (socket says 172.21.15.132:35890) 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f637fe640 1 -- 172.21.15.132:0/2512774558 learned_addr learned my addr 172.21.15.132:0/2512774558 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f627fc640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f64101ff0 0x7f2f64197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f62ffd640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f64100ea0 0x7f2f64197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f637fe640 1 -- 172.21.15.132:0/2512774558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f64100ea0 msgr2=0x7f2f64197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f637fe640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f64100ea0 0x7f2f64197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f637fe640 1 -- 172.21.15.132:0/2512774558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f64101ff0 msgr2=0x7f2f64197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f637fe640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f64101ff0 0x7f2f64197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f637fe640 1 -- 172.21.15.132:0/2512774558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6419ec70 con 0x7f2f641029b0 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f62ffd640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f64100ea0 0x7f2f64197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f627fc640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f64101ff0 0x7f2f64197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f637fe640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f641029b0 0x7f2f6419e600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00eaa0 tx=0x7f2f5c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f4bfff640 1 -- 172.21.15.132:0/2512774558 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00cdb0 con 0x7f2f641029b0 2024-04-23T18:34:01.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f4bfff640 1 -- 172.21.15.132:0/2512774558 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f5c005020 con 0x7f2f641029b0 2024-04-23T18:34:01.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f694cf640 1 -- 172.21.15.132:0/2512774558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6419ef60 con 0x7f2f641029b0 2024-04-23T18:34:01.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f4bfff640 1 -- 172.21.15.132:0/2512774558 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0116d0 con 0x7f2f641029b0 2024-04-23T18:34:01.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.638+0000 7f2f694cf640 1 -- 172.21.15.132:0/2512774558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6419f3d0 con 0x7f2f641029b0 2024-04-23T18:34:01.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.639+0000 7f2f694cf640 1 -- 172.21.15.132:0/2512774558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f641081b0 con 0x7f2f641029b0 2024-04-23T18:34:01.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.640+0000 7f2f4bfff640 1 -- 172.21.15.132:0/2512774558 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2f5c0051d0 con 0x7f2f641029b0 2024-04-23T18:34:01.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.641+0000 7f2f4bfff640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2f4c0a3bb0 0x7f2f4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.641+0000 7f2f62ffd640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2f4c0a3bb0 0x7f2f4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.642+0000 7f2f4bfff640 1 -- 172.21.15.132:0/2512774558 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f2f5c0d08e0 con 0x7f2f641029b0 2024-04-23T18:34:01.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.642+0000 7f2f62ffd640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2f4c0a3bb0 0x7f2f4c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2f54006010 tx=0x7f2f54000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:01.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.645+0000 7f2f4bfff640 1 -- 172.21.15.132:0/2512774558 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f5c016030 con 0x7f2f641029b0 2024-04-23T18:34:01.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.823+0000 7f2f694cf640 1 -- 172.21.15.132:0/2512774558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f6402d810 con 0x7f2f641029b0 2024-04-23T18:34:01.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.824+0000 7f2f4bfff640 1 -- 172.21.15.132:0/2512774558 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22314 (secure 0 0 0) 0x7f2f5c09d410 con 0x7f2f641029b0 2024-04-23T18:34:01.817 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:34:01.817 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":85,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:34:00.571380+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:34:01.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.826+0000 7f2f694cf640 1 -- 172.21.15.132:0/2512774558 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2f4c0a3bb0 msgr2=0x7f2f4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.826+0000 7f2f694cf640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2f4c0a3bb0 0x7f2f4c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2f54006010 tx=0x7f2f54000bc0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.826+0000 7f2f694cf640 1 -- 172.21.15.132:0/2512774558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f641029b0 msgr2=0x7f2f6419e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.826+0000 7f2f694cf640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f641029b0 0x7f2f6419e600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00eaa0 tx=0x7f2f5c00ef60 comp rx=0 tx=0).stop 2024-04-23T18:34:01.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.826+0000 7f2f694cf640 1 -- 172.21.15.132:0/2512774558 shutdown_connections 2024-04-23T18:34:01.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.826+0000 7f2f694cf640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2f4c0a3bb0 0x7f2f4c0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.826+0000 7f2f694cf640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f641029b0 0x7f2f6419e600 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.826+0000 7f2f694cf640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f64101ff0 0x7f2f64197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.826+0000 7f2f694cf640 1 --2- 172.21.15.132:0/2512774558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f64100ea0 0x7f2f64197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.826+0000 7f2f694cf640 1 -- 172.21.15.132:0/2512774558 >> 172.21.15.132:0/2512774558 conn(0x7f2f640fc430 msgr2=0x7f2f64105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:01.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.827+0000 7f2f694cf640 1 -- 172.21.15.132:0/2512774558 shutdown_connections 2024-04-23T18:34:01.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.827+0000 7f2f694cf640 1 -- 172.21.15.132:0/2512774558 wait complete. 2024-04-23T18:34:01.830 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:34:01.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.936+0000 7f5416cfd640 1 -- 172.21.15.132:0/1897528319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5410101fa0 msgr2=0x7f54101023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.936+0000 7f5416cfd640 1 --2- 172.21.15.132:0/1897528319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5410101fa0 0x7f54101023c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5404009900 tx=0x7f540402a410 comp rx=0 tx=0).stop 2024-04-23T18:34:01.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.936+0000 7f5416cfd640 1 -- 172.21.15.132:0/1897528319 shutdown_connections 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.936+0000 7f5416cfd640 1 --2- 172.21.15.132:0/1897528319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5410103190 0x7f5410103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.936+0000 7f5416cfd640 1 --2- 172.21.15.132:0/1897528319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5410103b50 0x7f541010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.936+0000 7f5416cfd640 1 --2- 172.21.15.132:0/1897528319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5410101fa0 0x7f54101023c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.936+0000 7f5416cfd640 1 -- 172.21.15.132:0/1897528319 >> 172.21.15.132:0/1897528319 conn(0x7f54100fd530 msgr2=0x7f54100ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.936+0000 7f5416cfd640 1 -- 172.21.15.132:0/1897528319 shutdown_connections 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.936+0000 7f5416cfd640 1 -- 172.21.15.132:0/1897528319 wait complete. 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.936+0000 7f5416cfd640 1 Processor -- start 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5416cfd640 1 -- start start 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5416cfd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5410101fa0 0x7f5410198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5416cfd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5410103190 0x7f54101987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5416cfd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5410103b50 0x7f541019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5416cfd640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f541010db40 con 0x7f5410101fa0 2024-04-23T18:34:01.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5416cfd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f541010d9c0 con 0x7f5410103b50 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5416cfd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f541010dcc0 con 0x7f5410103190 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5414a99640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5410101fa0 0x7f5410198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5414a99640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5410101fa0 0x7f5410198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35916/0 (socket says 172.21.15.132:35916) 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5414a99640 1 -- 172.21.15.132:0/2831592187 learned_addr learned my addr 172.21.15.132:0/2831592187 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f540ffff640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5410103190 0x7f54101987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f541529a640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5410103b50 0x7f541019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5414a99640 1 -- 172.21.15.132:0/2831592187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5410103190 msgr2=0x7f54101987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5414a99640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5410103190 0x7f54101987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5414a99640 1 -- 172.21.15.132:0/2831592187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5410103b50 msgr2=0x7f541019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5414a99640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5410103b50 0x7f541019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5414a99640 1 -- 172.21.15.132:0/2831592187 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541019fe80 con 0x7f5410101fa0 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f541529a640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5410103b50 0x7f541019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5414a99640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5410101fa0 0x7f5410198260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5404006010 tx=0x7f54040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f540dffb640 1 -- 172.21.15.132:0/2831592187 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540403f030 con 0x7f5410101fa0 2024-04-23T18:34:01.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f540dffb640 1 -- 172.21.15.132:0/2831592187 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5404003cb0 con 0x7f5410101fa0 2024-04-23T18:34:01.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f5416cfd640 1 -- 172.21.15.132:0/2831592187 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54101a0110 con 0x7f5410101fa0 2024-04-23T18:34:01.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.937+0000 7f540dffb640 1 -- 172.21.15.132:0/2831592187 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404035070 con 0x7f5410101fa0 2024-04-23T18:34:01.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.938+0000 7f5416cfd640 1 -- 172.21.15.132:0/2831592187 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54101a0680 con 0x7f5410101fa0 2024-04-23T18:34:01.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.939+0000 7f5416cfd640 1 -- 172.21.15.132:0/2831592187 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5410109390 con 0x7f5410101fa0 2024-04-23T18:34:01.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.940+0000 7f540dffb640 1 -- 172.21.15.132:0/2831592187 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5404042030 con 0x7f5410101fa0 2024-04-23T18:34:01.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.940+0000 7f540dffb640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f54000a3bb0 0x7f54000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.940+0000 7f540ffff640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f54000a3bb0 0x7f54000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.940+0000 7f540dffb640 1 -- 172.21.15.132:0/2831592187 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f54040f08f0 con 0x7f5410101fa0 2024-04-23T18:34:01.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.941+0000 7f540ffff640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f54000a3bb0 0x7f54000a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5410199770 tx=0x7f53fc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:01.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:01.945+0000 7f540dffb640 1 -- 172.21.15.132:0/2831592187 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54040bd420 con 0x7f5410101fa0 2024-04-23T18:34:02.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.124+0000 7f5416cfd640 1 -- 172.21.15.132:0/2831592187 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5410062b50 con 0x7f5410101fa0 2024-04-23T18:34:02.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.125+0000 7f540dffb640 1 -- 172.21.15.132:0/2831592187 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22314 (secure 0 0 0) 0x7f54040bcd20 con 0x7f5410101fa0 2024-04-23T18:34:02.118 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:34:02.118 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":85,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:34:00.571380+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:34:02.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.127+0000 7f5416cfd640 1 -- 172.21.15.132:0/2831592187 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f54000a3bb0 msgr2=0x7f54000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:02.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.127+0000 7f5416cfd640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f54000a3bb0 0x7f54000a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5410199770 tx=0x7f53fc00b390 comp rx=0 tx=0).stop 2024-04-23T18:34:02.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.127+0000 7f5416cfd640 1 -- 172.21.15.132:0/2831592187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5410101fa0 msgr2=0x7f5410198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:02.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.127+0000 7f5416cfd640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5410101fa0 0x7f5410198260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5404006010 tx=0x7f54040042f0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.128+0000 7f5416cfd640 1 -- 172.21.15.132:0/2831592187 shutdown_connections 2024-04-23T18:34:02.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.128+0000 7f5416cfd640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f54000a3bb0 0x7f54000a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.128+0000 7f5416cfd640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5410103190 0x7f54101987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.128+0000 7f5416cfd640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5410103b50 0x7f541019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.128+0000 7f5416cfd640 1 --2- 172.21.15.132:0/2831592187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5410101fa0 0x7f5410198260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.128+0000 7f5416cfd640 1 -- 172.21.15.132:0/2831592187 >> 172.21.15.132:0/2831592187 conn(0x7f54100fd530 msgr2=0x7f5410106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:02.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.128+0000 7f5416cfd640 1 -- 172.21.15.132:0/2831592187 shutdown_connections 2024-04-23T18:34:02.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.128+0000 7f5416cfd640 1 -- 172.21.15.132:0/2831592187 wait complete. 2024-04-23T18:34:02.131 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T18:34:02.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.242+0000 7f1732564640 1 -- 172.21.15.132:0/2924770134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f172c074930 msgr2=0x7f172c074d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:02.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.242+0000 7f1732564640 1 --2- 172.21.15.132:0/2924770134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f172c074930 0x7f172c074d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1720009900 tx=0x7f172002a480 comp rx=0 tx=0).stop 2024-04-23T18:34:02.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.242+0000 7f1732564640 1 -- 172.21.15.132:0/2924770134 shutdown_connections 2024-04-23T18:34:02.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.242+0000 7f1732564640 1 --2- 172.21.15.132:0/2924770134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f172c109e50 0x7f172c10c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.242+0000 7f1732564640 1 --2- 172.21.15.132:0/2924770134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f172c074930 0x7f172c074d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.242+0000 7f1732564640 1 --2- 172.21.15.132:0/2924770134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172c075f10 0x7f172c074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.242+0000 7f1732564640 1 -- 172.21.15.132:0/2924770134 >> 172.21.15.132:0/2924770134 conn(0x7f172c0fd530 msgr2=0x7f172c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.242+0000 7f1732564640 1 -- 172.21.15.132:0/2924770134 shutdown_connections 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.242+0000 7f1732564640 1 -- 172.21.15.132:0/2924770134 wait complete. 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.242+0000 7f1732564640 1 Processor -- start 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.242+0000 7f1732564640 1 -- start start 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f1732564640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f172c074930 0x7f172c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f1732564640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f172c075f10 0x7f172c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f1732564640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172c109e50 0x7f172c1a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f1732564640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f172c10fdb0 con 0x7f172c074930 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f1732564640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f172c10fc30 con 0x7f172c075f10 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f1732564640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f172c10ff30 con 0x7f172c109e50 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f172bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f172c074930 0x7f172c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f172bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f172c074930 0x7f172c19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35944/0 (socket says 172.21.15.132:35944) 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f172bfff640 1 -- 172.21.15.132:0/4013659822 learned_addr learned my addr 172.21.15.132:0/4013659822 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f172b7fe640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f172c075f10 0x7f172c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f172bfff640 1 -- 172.21.15.132:0/4013659822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172c109e50 msgr2=0x7f172c1a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:02.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f172bfff640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172c109e50 0x7f172c1a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f172bfff640 1 -- 172.21.15.132:0/4013659822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f172c075f10 msgr2=0x7f172c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:02.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f172bfff640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f172c075f10 0x7f172c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f172bfff640 1 -- 172.21.15.132:0/4013659822 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c1a21c0 con 0x7f172c074930 2024-04-23T18:34:02.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f172bfff640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f172c074930 0x7f172c19a5a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f171c00b4a0 tx=0x7f171c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:02.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f17297fa640 1 -- 172.21.15.132:0/4013659822 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c014070 con 0x7f172c074930 2024-04-23T18:34:02.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f17297fa640 1 -- 172.21.15.132:0/4013659822 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f171c010ac0 con 0x7f172c074930 2024-04-23T18:34:02.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f1732564640 1 -- 172.21.15.132:0/4013659822 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172c1a24b0 con 0x7f172c074930 2024-04-23T18:34:02.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f17297fa640 1 -- 172.21.15.132:0/4013659822 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c017c30 con 0x7f172c074930 2024-04-23T18:34:02.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.243+0000 7f1732564640 1 -- 172.21.15.132:0/4013659822 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172c1a2a80 con 0x7f172c074930 2024-04-23T18:34:02.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.244+0000 7f1732564640 1 -- 172.21.15.132:0/4013659822 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172c108c10 con 0x7f172c074930 2024-04-23T18:34:02.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.245+0000 7f17297fa640 1 -- 172.21.15.132:0/4013659822 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f171c017070 con 0x7f172c074930 2024-04-23T18:34:02.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.246+0000 7f17297fa640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f17140a3c30 0x7f17140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:02.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.246+0000 7f172b7fe640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f17140a3c30 0x7f17140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:02.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.246+0000 7f17297fa640 1 -- 172.21.15.132:0/4013659822 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f171c0cfa70 con 0x7f172c074930 2024-04-23T18:34:02.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.246+0000 7f172b7fe640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f17140a3c30 0x7f17140a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f172c19bab0 tx=0x7f17200314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:02.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.250+0000 7f17297fa640 1 -- 172.21.15.132:0/4013659822 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f171c09c580 con 0x7f172c074930 2024-04-23T18:34:02.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.468+0000 7f1732564640 1 -- 172.21.15.132:0/4013659822 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f172c02d5c0 con 0x7f172c074930 2024-04-23T18:34:02.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.519+0000 7f17297fa640 1 -- 172.21.15.132:0/4013659822 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f171c09be80 con 0x7f172c074930 2024-04-23T18:34:02.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.521+0000 7f1732564640 1 -- 172.21.15.132:0/4013659822 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f17140a3c30 msgr2=0x7f17140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:02.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.521+0000 7f1732564640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f17140a3c30 0x7f17140a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f172c19bab0 tx=0x7f17200314a0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.521+0000 7f1732564640 1 -- 172.21.15.132:0/4013659822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f172c074930 msgr2=0x7f172c19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:02.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.521+0000 7f1732564640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f172c074930 0x7f172c19a5a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f171c00b4a0 tx=0x7f171c00b960 comp rx=0 tx=0).stop 2024-04-23T18:34:02.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.522+0000 7f1732564640 1 -- 172.21.15.132:0/4013659822 shutdown_connections 2024-04-23T18:34:02.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.522+0000 7f1732564640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f17140a3c30 0x7f17140a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.522+0000 7f1732564640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172c109e50 0x7f172c1a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.522+0000 7f1732564640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f172c075f10 0x7f172c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.522+0000 7f1732564640 1 --2- 172.21.15.132:0/4013659822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f172c074930 0x7f172c19a5a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.522+0000 7f1732564640 1 -- 172.21.15.132:0/4013659822 >> 172.21.15.132:0/4013659822 conn(0x7f172c0fd530 msgr2=0x7f172c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:02.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.522+0000 7f1732564640 1 -- 172.21.15.132:0/4013659822 shutdown_connections 2024-04-23T18:34:02.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.522+0000 7f1732564640 1 -- 172.21.15.132:0/4013659822 wait complete. 2024-04-23T18:34:02.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:02 smithi148 ceph-mon[32311]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T18:34:02.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:02 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1575014349' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T18:34:02.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:02 smithi148 ceph-mon[32311]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T18:34:02.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:02 smithi148 ceph-mon[32311]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T18:34:02.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:02 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2512774558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:02.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:02 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2831592187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:02.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:02 smithi132 ceph-mon[29178]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T18:34:02.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:02 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1575014349' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T18:34:02.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:02 smithi132 ceph-mon[29178]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T18:34:02.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:02 smithi132 ceph-mon[29178]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T18:34:02.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:02 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2512774558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:02.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:02 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2831592187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:02.525 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-23T18:34:02.525 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-23T18:34:02.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:02 smithi157 ceph-mon[32580]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T18:34:02.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:02 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1575014349' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T18:34:02.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:02 smithi157 ceph-mon[32580]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T18:34:02.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:02 smithi157 ceph-mon[32580]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T18:34:02.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:02 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2512774558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:02.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:02 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2831592187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:02.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.634+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1789632471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b44102a10 msgr2=0x7f8b44104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:02.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.634+0000 7f8b4bc91640 1 --2- 172.21.15.132:0/1789632471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b44102a10 0x7f8b44104e00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009900 tx=0x7f8b4002a410 comp rx=0 tx=0).stop 2024-04-23T18:34:02.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.634+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1789632471 shutdown_connections 2024-04-23T18:34:02.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.634+0000 7f8b4bc91640 1 --2- 172.21.15.132:0/1789632471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b44107c70 0x7f8b4410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.634+0000 7f8b4bc91640 1 --2- 172.21.15.132:0/1789632471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44105340 0x7f8b44107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.634+0000 7f8b4bc91640 1 --2- 172.21.15.132:0/1789632471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b44102a10 0x7f8b44104e00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.634+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1789632471 >> 172.21.15.132:0/1789632471 conn(0x7f8b440fc400 msgr2=0x7f8b440fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:02.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.634+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1789632471 shutdown_connections 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.634+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1789632471 wait complete. 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4bc91640 1 Processor -- start 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4bc91640 1 -- start start 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4bc91640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b44102a10 0x7f8b44198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4bc91640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44105340 0x7f8b441987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4bc91640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b44107c70 0x7f8b4419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4bc91640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4410dae0 con 0x7f8b44107c70 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4bc91640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4410d960 con 0x7f8b44105340 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4bc91640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4410dc60 con 0x7f8b44102a10 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4a22e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b44107c70 0x7f8b4419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4a22e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b44107c70 0x7f8b4419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35972/0 (socket says 172.21.15.132:35972) 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4a22e640 1 -- 172.21.15.132:0/1436856066 learned_addr learned my addr 172.21.15.132:0/1436856066 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4922c640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44105340 0x7f8b441987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:02.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b49a2d640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b44102a10 0x7f8b44198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4a22e640 1 -- 172.21.15.132:0/1436856066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b44102a10 msgr2=0x7f8b44198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4a22e640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b44102a10 0x7f8b44198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4a22e640 1 -- 172.21.15.132:0/1436856066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44105340 msgr2=0x7f8b441987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4a22e640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44105340 0x7f8b441987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4a22e640 1 -- 172.21.15.132:0/1436856066 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4419fea0 con 0x7f8b44107c70 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.635+0000 7f8b4922c640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44105340 0x7f8b441987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.636+0000 7f8b4a22e640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b44107c70 0x7f8b4419f830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00cad0 tx=0x7f8b3c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.636+0000 7f8b3affd640 1 -- 172.21.15.132:0/1436856066 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00ae20 con 0x7f8b44107c70 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.636+0000 7f8b3affd640 1 -- 172.21.15.132:0/1436856066 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b3c005020 con 0x7f8b44107c70 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.636+0000 7f8b3affd640 1 -- 172.21.15.132:0/1436856066 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00f6b0 con 0x7f8b44107c70 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.636+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1436856066 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b441a0190 con 0x7f8b44107c70 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.636+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1436856066 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b441a0690 con 0x7f8b44107c70 2024-04-23T18:34:02.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.637+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1436856066 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b440fdef0 con 0x7f8b44107c70 2024-04-23T18:34:02.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.638+0000 7f8b3affd640 1 -- 172.21.15.132:0/1436856066 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8b3c0051d0 con 0x7f8b44107c70 2024-04-23T18:34:02.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.638+0000 7f8b3affd640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8b300a3c30 0x7f8b300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:02.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.638+0000 7f8b49a2d640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8b300a3c30 0x7f8b300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:02.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.638+0000 7f8b3affd640 1 -- 172.21.15.132:0/1436856066 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8b3c0d0310 con 0x7f8b44107c70 2024-04-23T18:34:02.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.639+0000 7f8b49a2d640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8b300a3c30 0x7f8b300a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8b40000c00 tx=0x7f8b400314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:02.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.643+0000 7f8b3affd640 1 -- 172.21.15.132:0/1436856066 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b3c014030 con 0x7f8b44107c70 2024-04-23T18:34:02.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:02.858+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1436856066 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f8b44062bf0 con 0x7f8b44107c70 2024-04-23T18:34:03.461 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:03 smithi157 ceph-mon[32580]: pgmap v289: 97 pgs: 21 creating+peering, 25 active+clean, 51 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:34:03.461 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:03 smithi157 ceph-mon[32580]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T18:34:03.461 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:03 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/4013659822' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T18:34:03.461 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:03 smithi157 ceph-mon[32580]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T18:34:03.461 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:03 smithi157 ceph-mon[32580]: Cluster is now healthy 2024-04-23T18:34:03.462 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:03 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/4013659822' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T18:34:03.462 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:03 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] up:active 2024-04-23T18:34:03.462 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:03 smithi157 ceph-mon[32580]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:34:03.462 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:03 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1436856066' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-23T18:34:03.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.521+0000 7f8b3affd640 1 -- 172.21.15.132:0/1436856066 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f8b3c09ce40 con 0x7f8b44107c70 2024-04-23T18:34:03.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:03 smithi148 ceph-mon[32311]: pgmap v289: 97 pgs: 21 creating+peering, 25 active+clean, 51 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:34:03.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:03 smithi148 ceph-mon[32311]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T18:34:03.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:03 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/4013659822' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T18:34:03.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:03 smithi148 ceph-mon[32311]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T18:34:03.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:03 smithi148 ceph-mon[32311]: Cluster is now healthy 2024-04-23T18:34:03.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:03 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/4013659822' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T18:34:03.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:03 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] up:active 2024-04-23T18:34:03.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:03 smithi148 ceph-mon[32311]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:34:03.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:03 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1436856066' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-23T18:34:03.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.524+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1436856066 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8b300a3c30 msgr2=0x7f8b300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:03.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.524+0000 7f8b4bc91640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8b300a3c30 0x7f8b300a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8b40000c00 tx=0x7f8b400314a0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.524+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1436856066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b44107c70 msgr2=0x7f8b4419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:03.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.524+0000 7f8b4bc91640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b44107c70 0x7f8b4419f830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00cad0 tx=0x7f8b3c00cf90 comp rx=0 tx=0).stop 2024-04-23T18:34:03.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.524+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1436856066 shutdown_connections 2024-04-23T18:34:03.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.524+0000 7f8b4bc91640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8b300a3c30 0x7f8b300a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.524+0000 7f8b4bc91640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b44107c70 0x7f8b4419f830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.524+0000 7f8b4bc91640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44105340 0x7f8b441987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.524+0000 7f8b4bc91640 1 --2- 172.21.15.132:0/1436856066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b44102a10 0x7f8b44198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.524+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1436856066 >> 172.21.15.132:0/1436856066 conn(0x7f8b440fc400 msgr2=0x7f8b441034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:03.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.524+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1436856066 shutdown_connections 2024-04-23T18:34:03.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.524+0000 7f8b4bc91640 1 -- 172.21.15.132:0/1436856066 wait complete. 2024-04-23T18:34:03.532 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:03 smithi132 ceph-mon[29178]: pgmap v289: 97 pgs: 21 creating+peering, 25 active+clean, 51 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:34:03.532 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:03 smithi132 ceph-mon[29178]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T18:34:03.532 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:03 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/4013659822' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T18:34:03.532 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:03 smithi132 ceph-mon[29178]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T18:34:03.532 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:03 smithi132 ceph-mon[29178]: Cluster is now healthy 2024-04-23T18:34:03.532 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:03 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/4013659822' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T18:34:03.532 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:03 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] up:active 2024-04-23T18:34:03.532 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:03 smithi132 ceph-mon[29178]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:34:03.532 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:03 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1436856066' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-23T18:34:03.533 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-23T18:34:03.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.645+0000 7f55bf7d8640 1 -- 172.21.15.132:0/2133911371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b8107cf0 msgr2=0x7f55b810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:03.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.645+0000 7f55bf7d8640 1 --2- 172.21.15.132:0/2133911371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b8107cf0 0x7f55b810a0e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009900 tx=0x7f55b002a410 comp rx=0 tx=0).stop 2024-04-23T18:34:03.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.645+0000 7f55bf7d8640 1 -- 172.21.15.132:0/2133911371 shutdown_connections 2024-04-23T18:34:03.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.645+0000 7f55bf7d8640 1 --2- 172.21.15.132:0/2133911371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b8107cf0 0x7f55b810a0e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.645+0000 7f55bf7d8640 1 --2- 172.21.15.132:0/2133911371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b81053c0 0x7f55b81077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.645+0000 7f55bf7d8640 1 --2- 172.21.15.132:0/2133911371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b8102a90 0x7f55b8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.645+0000 7f55bf7d8640 1 -- 172.21.15.132:0/2133911371 >> 172.21.15.132:0/2133911371 conn(0x7f55b80fc400 msgr2=0x7f55b80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.645+0000 7f55bf7d8640 1 -- 172.21.15.132:0/2133911371 shutdown_connections 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.645+0000 7f55bf7d8640 1 -- 172.21.15.132:0/2133911371 wait complete. 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bf7d8640 1 Processor -- start 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bf7d8640 1 -- start start 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bf7d8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b8102a90 0x7f55b8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bf7d8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b81053c0 0x7f55b81987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bf7d8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b8107cf0 0x7f55b819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bf7d8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f55b810dae0 con 0x7f55b8102a90 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bf7d8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55b810d960 con 0x7f55b81053c0 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bf7d8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f55b810dc60 con 0x7f55b8107cf0 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bd574640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b8102a90 0x7f55b8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bd574640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b8102a90 0x7f55b8198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35978/0 (socket says 172.21.15.132:35978) 2024-04-23T18:34:03.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bd574640 1 -- 172.21.15.132:0/3192166056 learned_addr learned my addr 172.21.15.132:0/3192166056 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bdd75640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b8107cf0 0x7f55b819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bcd73640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b81053c0 0x7f55b81987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bd574640 1 -- 172.21.15.132:0/3192166056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b8107cf0 msgr2=0x7f55b819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bd574640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b8107cf0 0x7f55b819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bd574640 1 -- 172.21.15.132:0/3192166056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b81053c0 msgr2=0x7f55b81987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bd574640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b81053c0 0x7f55b81987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bd574640 1 -- 172.21.15.132:0/3192166056 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b819fe50 con 0x7f55b8102a90 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.646+0000 7f55bcd73640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b81053c0 0x7f55b81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.647+0000 7f55bd574640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b8102a90 0x7f55b8198280 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f55b400eb90 tx=0x7f55b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.647+0000 7f55ae7fc640 1 -- 172.21.15.132:0/3192166056 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4004d20 con 0x7f55b8102a90 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.647+0000 7f55bf7d8640 1 -- 172.21.15.132:0/3192166056 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b81a0140 con 0x7f55b8102a90 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.647+0000 7f55ae7fc640 1 -- 172.21.15.132:0/3192166056 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55b400ee90 con 0x7f55b8102a90 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.647+0000 7f55ae7fc640 1 -- 172.21.15.132:0/3192166056 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4010670 con 0x7f55b8102a90 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.647+0000 7f55bf7d8640 1 -- 172.21.15.132:0/3192166056 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b81a0690 con 0x7f55b8102a90 2024-04-23T18:34:03.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.648+0000 7f55bf7d8640 1 -- 172.21.15.132:0/3192166056 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b80fdef0 con 0x7f55b8102a90 2024-04-23T18:34:03.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.649+0000 7f55ae7fc640 1 -- 172.21.15.132:0/3192166056 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f55b4005020 con 0x7f55b8102a90 2024-04-23T18:34:03.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.649+0000 7f55ae7fc640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f55a40a3bb0 0x7f55a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:03.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.649+0000 7f55bcd73640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f55a40a3bb0 0x7f55a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:03.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.650+0000 7f55ae7fc640 1 -- 172.21.15.132:0/3192166056 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f55b40d0950 con 0x7f55b8102a90 2024-04-23T18:34:03.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.650+0000 7f55bcd73640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f55a40a3bb0 0x7f55a40a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55a80058c0 tx=0x7f55a8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:03.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.654+0000 7f55ae7fc640 1 -- 172.21.15.132:0/3192166056 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55b409d480 con 0x7f55b8102a90 2024-04-23T18:34:03.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:03.869+0000 7f55bf7d8640 1 -- 172.21.15.132:0/3192166056 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f55b8062b50 con 0x7f55b8102a90 2024-04-23T18:34:04.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:04 smithi148 ceph-mon[32311]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T18:34:04.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:04 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1436856066' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-23T18:34:04.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:04 smithi148 ceph-mon[32311]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T18:34:04.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:04 smithi148 ceph-mon[32311]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:34:04.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:04 smithi148 ceph-mon[32311]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-23T18:34:04.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:04 smithi148 ceph-mon[32311]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-23T18:34:04.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:04 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3192166056' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T18:34:04.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.531+0000 7f55ae7fc640 1 -- 172.21.15.132:0/3192166056 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f55b409cd80 con 0x7f55b8102a90 2024-04-23T18:34:04.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.534+0000 7f55bf7d8640 1 -- 172.21.15.132:0/3192166056 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f55a40a3bb0 msgr2=0x7f55a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.534+0000 7f55bf7d8640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f55a40a3bb0 0x7f55a40a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55a80058c0 tx=0x7f55a8005850 comp rx=0 tx=0).stop 2024-04-23T18:34:04.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.534+0000 7f55bf7d8640 1 -- 172.21.15.132:0/3192166056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b8102a90 msgr2=0x7f55b8198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.534+0000 7f55bf7d8640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b8102a90 0x7f55b8198280 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f55b400eb90 tx=0x7f55b400c6a0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.534+0000 7f55bf7d8640 1 -- 172.21.15.132:0/3192166056 shutdown_connections 2024-04-23T18:34:04.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.534+0000 7f55bf7d8640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f55a40a3bb0 0x7f55a40a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.534+0000 7f55bf7d8640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55b8107cf0 0x7f55b819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.534+0000 7f55bf7d8640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b81053c0 0x7f55b81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.534+0000 7f55bf7d8640 1 --2- 172.21.15.132:0/3192166056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b8102a90 0x7f55b8198280 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.534+0000 7f55bf7d8640 1 -- 172.21.15.132:0/3192166056 >> 172.21.15.132:0/3192166056 conn(0x7f55b80fc400 msgr2=0x7f55b81035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:04.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.535+0000 7f55bf7d8640 1 -- 172.21.15.132:0/3192166056 shutdown_connections 2024-04-23T18:34:04.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.535+0000 7f55bf7d8640 1 -- 172.21.15.132:0/3192166056 wait complete. 2024-04-23T18:34:04.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:04 smithi157 ceph-mon[32580]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T18:34:04.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:04 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1436856066' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-23T18:34:04.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:04 smithi157 ceph-mon[32580]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T18:34:04.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:04 smithi157 ceph-mon[32580]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:34:04.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:04 smithi157 ceph-mon[32580]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-23T18:34:04.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:04 smithi157 ceph-mon[32580]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-23T18:34:04.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:04 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3192166056' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T18:34:04.542 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:04 smithi132 ceph-mon[29178]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T18:34:04.542 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:04 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1436856066' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-23T18:34:04.542 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:04 smithi132 ceph-mon[29178]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T18:34:04.542 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:04 smithi132 ceph-mon[29178]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:34:04.542 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:04 smithi132 ceph-mon[29178]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-23T18:34:04.542 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:04 smithi132 ceph-mon[29178]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-23T18:34:04.542 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:04 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3192166056' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T18:34:04.543 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-23T18:34:04.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.657+0000 7f96ec031640 1 -- 172.21.15.132:0/2126991812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96e4101fa0 msgr2=0x7f96e41023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.657+0000 7f96ec031640 1 --2- 172.21.15.132:0/2126991812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96e4101fa0 0x7f96e41023c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009900 tx=0x7f96e002a410 comp rx=0 tx=0).stop 2024-04-23T18:34:04.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.657+0000 7f96ec031640 1 -- 172.21.15.132:0/2126991812 shutdown_connections 2024-04-23T18:34:04.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.657+0000 7f96ec031640 1 --2- 172.21.15.132:0/2126991812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4103190 0x7f96e4103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.657+0000 7f96ec031640 1 --2- 172.21.15.132:0/2126991812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96e4103b50 0x7f96e410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.657+0000 7f96ec031640 1 --2- 172.21.15.132:0/2126991812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96e4101fa0 0x7f96e41023c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.657+0000 7f96ec031640 1 -- 172.21.15.132:0/2126991812 >> 172.21.15.132:0/2126991812 conn(0x7f96e40fd530 msgr2=0x7f96e40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.657+0000 7f96ec031640 1 -- 172.21.15.132:0/2126991812 shutdown_connections 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.657+0000 7f96ec031640 1 -- 172.21.15.132:0/2126991812 wait complete. 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.657+0000 7f96ec031640 1 Processor -- start 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.657+0000 7f96ec031640 1 -- start start 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ec031640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4101fa0 0x7f96e4075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ec031640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96e4103190 0x7f96e4074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ec031640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96e4103b50 0x7f96e4074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ec031640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f96e410dae0 con 0x7f96e4103b50 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ec031640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96e410d960 con 0x7f96e4103190 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ec031640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96e410dc60 con 0x7f96e4101fa0 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ea5ce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96e4103b50 0x7f96e4074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ea5ce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96e4103b50 0x7f96e4074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35996/0 (socket says 172.21.15.132:35996) 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ea5ce640 1 -- 172.21.15.132:0/2315798497 learned_addr learned my addr 172.21.15.132:0/2315798497 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96e9dcd640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4101fa0 0x7f96e4075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96e95cc640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96e4103190 0x7f96e4074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ea5ce640 1 -- 172.21.15.132:0/2315798497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4101fa0 msgr2=0x7f96e4075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ea5ce640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4101fa0 0x7f96e4075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ea5ce640 1 -- 172.21.15.132:0/2315798497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96e4103190 msgr2=0x7f96e4074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ea5ce640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96e4103190 0x7f96e4074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ea5ce640 1 -- 172.21.15.132:0/2315798497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e4075110 con 0x7f96e4103b50 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ea5ce640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96e4103b50 0x7f96e4074880 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00eb10 tx=0x7f96dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96daffd640 1 -- 172.21.15.132:0/2315798497 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc004b20 con 0x7f96e4103b50 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ec031640 1 -- 172.21.15.132:0/2315798497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e41a2310 con 0x7f96e4103b50 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96daffd640 1 -- 172.21.15.132:0/2315798497 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96dc004cc0 con 0x7f96e4103b50 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96daffd640 1 -- 172.21.15.132:0/2315798497 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc010670 con 0x7f96e4103b50 2024-04-23T18:34:04.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.658+0000 7f96ec031640 1 -- 172.21.15.132:0/2315798497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e41a2860 con 0x7f96e4103b50 2024-04-23T18:34:04.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.659+0000 7f96ec031640 1 -- 172.21.15.132:0/2315798497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e4109380 con 0x7f96e4103b50 2024-04-23T18:34:04.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.660+0000 7f96daffd640 1 -- 172.21.15.132:0/2315798497 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f96dc005000 con 0x7f96e4103b50 2024-04-23T18:34:04.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.661+0000 7f96daffd640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f96d00a3bb0 0x7f96d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:04.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.661+0000 7f96e9dcd640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f96d00a3bb0 0x7f96d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:04.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.661+0000 7f96daffd640 1 -- 172.21.15.132:0/2315798497 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f96dc0d0ab0 con 0x7f96e4103b50 2024-04-23T18:34:04.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.662+0000 7f96e9dcd640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f96d00a3bb0 0x7f96d00a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009230 tx=0x7f96e00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:04.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.665+0000 7f96daffd640 1 -- 172.21.15.132:0/2315798497 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96dc053ff0 con 0x7f96e4103b50 2024-04-23T18:34:04.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:04.881+0000 7f96ec031640 1 -- 172.21.15.132:0/2315798497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f96e4062b50 con 0x7f96e4103b50 2024-04-23T18:34:05.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.515+0000 7f96daffd640 1 -- 172.21.15.132:0/2315798497 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f96dc09d580 con 0x7f96e4103b50 2024-04-23T18:34:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.517+0000 7f96ec031640 1 -- 172.21.15.132:0/2315798497 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f96d00a3bb0 msgr2=0x7f96d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.517+0000 7f96ec031640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f96d00a3bb0 0x7f96d00a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009230 tx=0x7f96e00314a0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.517+0000 7f96ec031640 1 -- 172.21.15.132:0/2315798497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96e4103b50 msgr2=0x7f96e4074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.517+0000 7f96ec031640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96e4103b50 0x7f96e4074880 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00eb10 tx=0x7f96dc00c2d0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.517+0000 7f96ec031640 1 -- 172.21.15.132:0/2315798497 shutdown_connections 2024-04-23T18:34:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.517+0000 7f96ec031640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f96d00a3bb0 0x7f96d00a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.517+0000 7f96ec031640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96e4103b50 0x7f96e4074880 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.517+0000 7f96ec031640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96e4103190 0x7f96e4074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.517+0000 7f96ec031640 1 --2- 172.21.15.132:0/2315798497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e4101fa0 0x7f96e4075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.517+0000 7f96ec031640 1 -- 172.21.15.132:0/2315798497 >> 172.21.15.132:0/2315798497 conn(0x7f96e40fd530 msgr2=0x7f96e4106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.517+0000 7f96ec031640 1 -- 172.21.15.132:0/2315798497 shutdown_connections 2024-04-23T18:34:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.517+0000 7f96ec031640 1 -- 172.21.15.132:0/2315798497 wait complete. 2024-04-23T18:34:05.514 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:34:05 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b[62525]: 2024-04-23T18:34:05.186+0000 7f3555644700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T18:34:05.514 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:34:05 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b[62525]: 2024-04-23T18:34:05.186+0000 7f3555644700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-23T18:34:05.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:05 smithi148 ceph-mon[32311]: pgmap v291: 97 pgs: 21 creating+peering, 46 active+clean, 30 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 208 B/s wr, 4 op/s 2024-04-23T18:34:05.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:05 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3192166056' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T18:34:05.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:05 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] up:active 2024-04-23T18:34:05.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:05 smithi148 ceph-mon[32311]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T18:34:05.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:05 smithi148 ceph-mon[32311]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-23T18:34:05.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:05 smithi148 ceph-mon[32311]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-23T18:34:05.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:05 smithi148 ceph-mon[32311]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-23T18:34:05.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:05 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2315798497' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T18:34:05.522 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-mon[29178]: pgmap v291: 97 pgs: 21 creating+peering, 46 active+clean, 30 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 208 B/s wr, 4 op/s 2024-04-23T18:34:05.523 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3192166056' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T18:34:05.523 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] up:active 2024-04-23T18:34:05.523 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-mon[29178]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T18:34:05.523 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-mon[29178]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-23T18:34:05.523 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-mon[29178]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-23T18:34:05.523 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-mon[29178]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-23T18:34:05.523 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2315798497' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T18:34:05.524 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-23T18:34:05.524 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-23T18:34:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:05 smithi157 ceph-mon[32580]: pgmap v291: 97 pgs: 21 creating+peering, 46 active+clean, 30 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 208 B/s wr, 4 op/s 2024-04-23T18:34:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:05 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3192166056' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T18:34:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:05 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] up:active 2024-04-23T18:34:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:05 smithi157 ceph-mon[32580]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T18:34:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:05 smithi157 ceph-mon[32580]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-23T18:34:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:05 smithi157 ceph-mon[32580]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-23T18:34:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:05 smithi157 ceph-mon[32580]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-23T18:34:05.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:05 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2315798497' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T18:34:05.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.633+0000 7f481fa2a640 1 -- 172.21.15.132:0/3931831209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4818103230 msgr2=0x7f48181036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.633+0000 7f481fa2a640 1 --2- 172.21.15.132:0/3931831209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4818103230 0x7f48181036b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4808009930 tx=0x7f480802a420 comp rx=0 tx=0).stop 2024-04-23T18:34:05.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.633+0000 7f481fa2a640 1 -- 172.21.15.132:0/3931831209 shutdown_connections 2024-04-23T18:34:05.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.633+0000 7f481fa2a640 1 --2- 172.21.15.132:0/3931831209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4818103bf0 0x7f481810a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.633+0000 7f481fa2a640 1 --2- 172.21.15.132:0/3931831209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4818103230 0x7f48181036b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.633+0000 7f481fa2a640 1 --2- 172.21.15.132:0/3931831209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4818100260 0x7f4818100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.633+0000 7f481fa2a640 1 -- 172.21.15.132:0/3931831209 >> 172.21.15.132:0/3931831209 conn(0x7f48180fbb90 msgr2=0x7f48180fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:05.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.633+0000 7f481fa2a640 1 -- 172.21.15.132:0/3931831209 shutdown_connections 2024-04-23T18:34:05.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.633+0000 7f481fa2a640 1 -- 172.21.15.132:0/3931831209 wait complete. 2024-04-23T18:34:05.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481fa2a640 1 Processor -- start 2024-04-23T18:34:05.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481fa2a640 1 -- start start 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481fa2a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4818100260 0x7f4818198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481fa2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4818103230 0x7f4818198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481fa2a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4818103bf0 0x7f481819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481fa2a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f481810db50 con 0x7f4818100260 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481fa2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f481810d9d0 con 0x7f4818103230 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481fa2a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f481810dcd0 con 0x7f4818103bf0 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481d7c6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4818100260 0x7f4818198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481d7c6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4818100260 0x7f4818198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36004/0 (socket says 172.21.15.132:36004) 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481d7c6640 1 -- 172.21.15.132:0/2536765498 learned_addr learned my addr 172.21.15.132:0/2536765498 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481cfc5640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4818103230 0x7f4818198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481dfc7640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4818103bf0 0x7f481819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481d7c6640 1 -- 172.21.15.132:0/2536765498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4818103bf0 msgr2=0x7f481819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481d7c6640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4818103bf0 0x7f481819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481d7c6640 1 -- 172.21.15.132:0/2536765498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4818103230 msgr2=0x7f4818198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481d7c6640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4818103230 0x7f4818198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481d7c6640 1 -- 172.21.15.132:0/2536765498 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481819fe40 con 0x7f4818100260 2024-04-23T18:34:05.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481cfc5640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4818103230 0x7f4818198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:05.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f481d7c6640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4818100260 0x7f4818198220 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f481400e9c0 tx=0x7f481400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:05.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.634+0000 7f480e7fc640 1 -- 172.21.15.132:0/2536765498 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481400cd80 con 0x7f4818100260 2024-04-23T18:34:05.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.635+0000 7f480e7fc640 1 -- 172.21.15.132:0/2536765498 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48140035f0 con 0x7f4818100260 2024-04-23T18:34:05.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.635+0000 7f481fa2a640 1 -- 172.21.15.132:0/2536765498 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48181a0130 con 0x7f4818100260 2024-04-23T18:34:05.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.635+0000 7f480e7fc640 1 -- 172.21.15.132:0/2536765498 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814010690 con 0x7f4818100260 2024-04-23T18:34:05.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.635+0000 7f481fa2a640 1 -- 172.21.15.132:0/2536765498 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48181a0650 con 0x7f4818100260 2024-04-23T18:34:05.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.636+0000 7f481fa2a640 1 -- 172.21.15.132:0/2536765498 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48181093e0 con 0x7f4818100260 2024-04-23T18:34:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.637+0000 7f480e7fc640 1 -- 172.21.15.132:0/2536765498 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f48140037a0 con 0x7f4818100260 2024-04-23T18:34:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.637+0000 7f480e7fc640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f48040a3bb0 0x7f48040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.637+0000 7f481cfc5640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f48040a3bb0 0x7f48040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.637+0000 7f480e7fc640 1 -- 172.21.15.132:0/2536765498 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f48140d0860 con 0x7f4818100260 2024-04-23T18:34:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.638+0000 7f481cfc5640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f48040a3bb0 0x7f48040a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4808006010 tx=0x7f48080314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:05.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.641+0000 7f480e7fc640 1 -- 172.21.15.132:0/2536765498 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f481401f340 con 0x7f4818100260 2024-04-23T18:34:05.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:05.845+0000 7f481fa2a640 1 -- 172.21.15.132:0/2536765498 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4818062390 con 0x7f48040a3bb0 2024-04-23T18:34:06.111 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:34:05.850+0000 7f6bba9c6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:34:06.111 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:34:05.850+0000 7f6bba9c6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:34:06.111 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:34:05.850+0000 7f6bba9c6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:34:06.111 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:34:05.850+0000 7f6bba9c6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:34:06.111 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:34:05 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:34:05.850+0000 7f6bba9c6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:34:06.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.547+0000 7f480e7fc640 1 -- 172.21.15.132:0/2536765498 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4818062390 con 0x7f48040a3bb0 2024-04-23T18:34:06.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.549+0000 7f481fa2a640 1 -- 172.21.15.132:0/2536765498 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f48040a3bb0 msgr2=0x7f48040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:06.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.549+0000 7f481fa2a640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f48040a3bb0 0x7f48040a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4808006010 tx=0x7f48080314a0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.549+0000 7f481fa2a640 1 -- 172.21.15.132:0/2536765498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4818100260 msgr2=0x7f4818198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:06.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.549+0000 7f481fa2a640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4818100260 0x7f4818198220 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f481400e9c0 tx=0x7f481400ee80 comp rx=0 tx=0).stop 2024-04-23T18:34:06.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.549+0000 7f481fa2a640 1 -- 172.21.15.132:0/2536765498 shutdown_connections 2024-04-23T18:34:06.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.549+0000 7f481fa2a640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f48040a3bb0 0x7f48040a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.549+0000 7f481fa2a640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4818103bf0 0x7f481819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.549+0000 7f481fa2a640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4818103230 0x7f4818198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.550+0000 7f481fa2a640 1 --2- 172.21.15.132:0/2536765498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4818100260 0x7f4818198220 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.550+0000 7f481fa2a640 1 -- 172.21.15.132:0/2536765498 >> 172.21.15.132:0/2536765498 conn(0x7f48180fbb90 msgr2=0x7f48181065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:06.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.550+0000 7f481fa2a640 1 -- 172.21.15.132:0/2536765498 shutdown_connections 2024-04-23T18:34:06.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.550+0000 7f481fa2a640 1 -- 172.21.15.132:0/2536765498 wait complete. 2024-04-23T18:34:06.555 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-23T18:34:06.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.671+0000 7fd03d35c640 1 -- 172.21.15.132:0/1507317240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd038105d60 msgr2=0x7fd03810c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.671+0000 7fd03d35c640 1 --2- 172.21.15.132:0/1507317240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd038105d60 0x7fd03810c5f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd030009ca0 tx=0x7fd03002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.671+0000 7fd03d35c640 1 -- 172.21.15.132:0/1507317240 shutdown_connections 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.671+0000 7fd03d35c640 1 --2- 172.21.15.132:0/1507317240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd038105d60 0x7fd03810c5f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.671+0000 7fd03d35c640 1 --2- 172.21.15.132:0/1507317240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0381053a0 0x7fd038105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.671+0000 7fd03d35c640 1 --2- 172.21.15.132:0/1507317240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0381041b0 0x7fd0381045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.671+0000 7fd03d35c640 1 -- 172.21.15.132:0/1507317240 >> 172.21.15.132:0/1507317240 conn(0x7fd0380ff740 msgr2=0x7fd038101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.671+0000 7fd03d35c640 1 -- 172.21.15.132:0/1507317240 shutdown_connections 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.671+0000 7fd03d35c640 1 -- 172.21.15.132:0/1507317240 wait complete. 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd03d35c640 1 Processor -- start 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd03d35c640 1 -- start start 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd03d35c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0381041b0 0x7fd03819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd03d35c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0381053a0 0x7fd03819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:06.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd03d35c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd038105d60 0x7fd0381a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd03d35c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd03810fcf0 con 0x7fd0381053a0 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd03d35c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd03810fb70 con 0x7fd0381041b0 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd03d35c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd03810fe70 con 0x7fd038105d60 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd0367fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0381053a0 0x7fd03819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd0367fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0381053a0 0x7fd03819a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36044/0 (socket says 172.21.15.132:36044) 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd0367fc640 1 -- 172.21.15.132:0/2580961211 learned_addr learned my addr 172.21.15.132:0/2580961211 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd036ffd640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0381041b0 0x7fd03819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd0377fe640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd038105d60 0x7fd0381a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd0367fc640 1 -- 172.21.15.132:0/2580961211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd038105d60 msgr2=0x7fd0381a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd0367fc640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd038105d60 0x7fd0381a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd0367fc640 1 -- 172.21.15.132:0/2580961211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0381041b0 msgr2=0x7fd03819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd0367fc640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0381041b0 0x7fd03819a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd0367fc640 1 -- 172.21.15.132:0/2580961211 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0381a1fc0 con 0x7fd0381053a0 2024-04-23T18:34:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.672+0000 7fd0377fe640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd038105d60 0x7fd0381a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.673+0000 7fd0367fc640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0381053a0 0x7fd03819a8e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00ea60 tx=0x7fd02c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.673+0000 7fd01ffff640 1 -- 172.21.15.132:0/2580961211 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c00cde0 con 0x7fd0381053a0 2024-04-23T18:34:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.673+0000 7fd01ffff640 1 -- 172.21.15.132:0/2580961211 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd02c0035f0 con 0x7fd0381053a0 2024-04-23T18:34:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.673+0000 7fd03d35c640 1 -- 172.21.15.132:0/2580961211 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0381a22b0 con 0x7fd0381053a0 2024-04-23T18:34:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.673+0000 7fd01ffff640 1 -- 172.21.15.132:0/2580961211 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c010690 con 0x7fd0381053a0 2024-04-23T18:34:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.673+0000 7fd03d35c640 1 -- 172.21.15.132:0/2580961211 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0381a2800 con 0x7fd0381053a0 2024-04-23T18:34:06.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.674+0000 7fd03d35c640 1 -- 172.21.15.132:0/2580961211 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd03810b550 con 0x7fd0381053a0 2024-04-23T18:34:06.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.675+0000 7fd01ffff640 1 -- 172.21.15.132:0/2580961211 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd02c0037a0 con 0x7fd0381053a0 2024-04-23T18:34:06.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.675+0000 7fd01ffff640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd0200a3bb0 0x7fd0200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:06.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.675+0000 7fd036ffd640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd0200a3bb0 0x7fd0200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:06.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.676+0000 7fd01ffff640 1 -- 172.21.15.132:0/2580961211 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd02c0d05f0 con 0x7fd0381053a0 2024-04-23T18:34:06.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.676+0000 7fd036ffd640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd0200a3bb0 0x7fd0200a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd028005fd0 tx=0x7fd02800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:06.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.680+0000 7fd01ffff640 1 -- 172.21.15.132:0/2580961211 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd02c09d120 con 0x7fd0381053a0 2024-04-23T18:34:06.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:06 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2315798497' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T18:34:06.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:06 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] up:active 2024-04-23T18:34:06.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:06 smithi148 ceph-mon[32311]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-23T18:34:06.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:06 smithi148 ceph-mon[32311]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:06.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:06 smithi148 ceph-mon[32311]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-23T18:34:06.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:06 smithi148 ceph-mon[32311]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-23T18:34:06.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:06 smithi148 ceph-mon[32311]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:06.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:06 smithi148 ceph-mon[32311]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s wr, 15 op/s 2024-04-23T18:34:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:06 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2315798497' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T18:34:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:06 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] up:active 2024-04-23T18:34:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:06 smithi157 ceph-mon[32580]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-23T18:34:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:06 smithi157 ceph-mon[32580]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:06 smithi157 ceph-mon[32580]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-23T18:34:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:06 smithi157 ceph-mon[32580]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-23T18:34:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:06 smithi157 ceph-mon[32580]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:06 smithi157 ceph-mon[32580]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s wr, 15 op/s 2024-04-23T18:34:06.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:06 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2315798497' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T18:34:06.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:06 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] up:active 2024-04-23T18:34:06.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:06 smithi132 ceph-mon[29178]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-23T18:34:06.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:06 smithi132 ceph-mon[29178]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:06.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:06 smithi132 ceph-mon[29178]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-23T18:34:06.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:06 smithi132 ceph-mon[29178]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-23T18:34:06.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:06 smithi132 ceph-mon[29178]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:06.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:06 smithi132 ceph-mon[29178]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s wr, 15 op/s 2024-04-23T18:34:06.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.886+0000 7fd03d35c640 1 -- 172.21.15.132:0/2580961211 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd038062390 con 0x7fd0200a3bb0 2024-04-23T18:34:06.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.909+0000 7fd01ffff640 1 -- 172.21.15.132:0/2580961211 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd038062390 con 0x7fd0200a3bb0 2024-04-23T18:34:06.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.911+0000 7fd03d35c640 1 -- 172.21.15.132:0/2580961211 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd0200a3bb0 msgr2=0x7fd0200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:06.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.911+0000 7fd03d35c640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd0200a3bb0 0x7fd0200a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd028005fd0 tx=0x7fd02800b390 comp rx=0 tx=0).stop 2024-04-23T18:34:06.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.911+0000 7fd03d35c640 1 -- 172.21.15.132:0/2580961211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0381053a0 msgr2=0x7fd03819a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:06.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.911+0000 7fd03d35c640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0381053a0 0x7fd03819a8e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00ea60 tx=0x7fd02c00ef20 comp rx=0 tx=0).stop 2024-04-23T18:34:06.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.911+0000 7fd03d35c640 1 -- 172.21.15.132:0/2580961211 shutdown_connections 2024-04-23T18:34:06.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.911+0000 7fd03d35c640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd0200a3bb0 0x7fd0200a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.911+0000 7fd03d35c640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd038105d60 0x7fd0381a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.911+0000 7fd03d35c640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0381053a0 0x7fd03819a8e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.911+0000 7fd03d35c640 1 --2- 172.21.15.132:0/2580961211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0381041b0 0x7fd03819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:06.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.911+0000 7fd03d35c640 1 -- 172.21.15.132:0/2580961211 >> 172.21.15.132:0/2580961211 conn(0x7fd0380ff740 msgr2=0x7fd038108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:06.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.911+0000 7fd03d35c640 1 -- 172.21.15.132:0/2580961211 shutdown_connections 2024-04-23T18:34:06.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:06.911+0000 7fd03d35c640 1 -- 172.21.15.132:0/2580961211 wait complete. 2024-04-23T18:34:06.914 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-23T18:34:07.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 -- 172.21.15.132:0/3307501831 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4a4102040 msgr2=0x7fd4a41024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 --2- 172.21.15.132:0/3307501831 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4a4102040 0x7fd4a41024c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009900 tx=0x7fd48c02a410 comp rx=0 tx=0).stop 2024-04-23T18:34:07.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 -- 172.21.15.132:0/3307501831 shutdown_connections 2024-04-23T18:34:07.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 --2- 172.21.15.132:0/3307501831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a4102a00 0x7fd4a4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 --2- 172.21.15.132:0/3307501831 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4a4102040 0x7fd4a41024c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 --2- 172.21.15.132:0/3307501831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4a4100e60 0x7fd4a4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 -- 172.21.15.132:0/3307501831 >> 172.21.15.132:0/3307501831 conn(0x7fd4a40fc420 msgr2=0x7fd4a40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 -- 172.21.15.132:0/3307501831 shutdown_connections 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 -- 172.21.15.132:0/3307501831 wait complete. 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 Processor -- start 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 -- start start 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4a4100e60 0x7fd4a406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.025+0000 7fd4aadc4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a4102040 0x7fd4a419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4aadc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4a4102a00 0x7fd4a419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4aadc4640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a410ca00 con 0x7fd4a4100e60 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4aadc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a410c880 con 0x7fd4a4102a00 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4aadc4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a410cb80 con 0x7fd4a4102040 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a8b60640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4a4100e60 0x7fd4a406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a8b60640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4a4100e60 0x7fd4a406b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36060/0 (socket says 172.21.15.132:36060) 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a8b60640 1 -- 172.21.15.132:0/2495819871 learned_addr learned my addr 172.21.15.132:0/2495819871 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a9361640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4a4102a00 0x7fd4a419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:07.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a3fff640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a4102040 0x7fd4a419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:07.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a8b60640 1 -- 172.21.15.132:0/2495819871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a4102040 msgr2=0x7fd4a419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a8b60640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a4102040 0x7fd4a419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a8b60640 1 -- 172.21.15.132:0/2495819871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4a4102a00 msgr2=0x7fd4a419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a8b60640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4a4102a00 0x7fd4a419f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a8b60640 1 -- 172.21.15.132:0/2495819871 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a419fe80 con 0x7fd4a4100e60 2024-04-23T18:34:07.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a8b60640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4a4100e60 0x7fd4a406b0c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd49400e9c0 tx=0x7fd49400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:07.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a1ffb640 1 -- 172.21.15.132:0/2495819871 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49400cd80 con 0x7fd4a4100e60 2024-04-23T18:34:07.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a1ffb640 1 -- 172.21.15.132:0/2495819871 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4940035f0 con 0x7fd4a4100e60 2024-04-23T18:34:07.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4aadc4640 1 -- 172.21.15.132:0/2495819871 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a41a0110 con 0x7fd4a4100e60 2024-04-23T18:34:07.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4a1ffb640 1 -- 172.21.15.132:0/2495819871 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494010690 con 0x7fd4a4100e60 2024-04-23T18:34:07.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.026+0000 7fd4aadc4640 1 -- 172.21.15.132:0/2495819871 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a41a06b0 con 0x7fd4a4100e60 2024-04-23T18:34:07.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.028+0000 7fd4aadc4640 1 -- 172.21.15.132:0/2495819871 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a41082d0 con 0x7fd4a4100e60 2024-04-23T18:34:07.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.028+0000 7fd4a1ffb640 1 -- 172.21.15.132:0/2495819871 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd4940037a0 con 0x7fd4a4100e60 2024-04-23T18:34:07.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.029+0000 7fd4a1ffb640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd4900a3c30 0x7fd4900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:07.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.029+0000 7fd4a3fff640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd4900a3c30 0x7fd4900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:07.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.029+0000 7fd4a1ffb640 1 -- 172.21.15.132:0/2495819871 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd4940d06e0 con 0x7fd4a4100e60 2024-04-23T18:34:07.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.030+0000 7fd4a3fff640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd4900a3c30 0x7fd4900a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd48c02a930 tx=0x7fd48c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:07.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.034+0000 7fd4a1ffb640 1 -- 172.21.15.132:0/2495819871 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd49409d170 con 0x7fd4a4100e60 2024-04-23T18:34:07.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.239+0000 7fd4aadc4640 1 -- 172.21.15.132:0/2495819871 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fd4a4062420 con 0x7fd4900a3c30 2024-04-23T18:34:07.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.244+0000 7fd4a1ffb640 1 -- 172.21.15.132:0/2495819871 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd4a4062420 con 0x7fd4900a3c30 2024-04-23T18:34:07.236 INFO:teuthology.orchestra.run.smithi132.stdout:/volumes/qa/sv_0/e48570be-639f-4020-84da-07c4da574626 2024-04-23T18:34:07.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.246+0000 7fd4aadc4640 1 -- 172.21.15.132:0/2495819871 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd4900a3c30 msgr2=0x7fd4900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.246+0000 7fd4aadc4640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd4900a3c30 0x7fd4900a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd48c02a930 tx=0x7fd48c000bc0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.246+0000 7fd4aadc4640 1 -- 172.21.15.132:0/2495819871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4a4100e60 msgr2=0x7fd4a406b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.246+0000 7fd4aadc4640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4a4100e60 0x7fd4a406b0c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd49400e9c0 tx=0x7fd49400ee80 comp rx=0 tx=0).stop 2024-04-23T18:34:07.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.246+0000 7fd4aadc4640 1 -- 172.21.15.132:0/2495819871 shutdown_connections 2024-04-23T18:34:07.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.246+0000 7fd4aadc4640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd4900a3c30 0x7fd4900a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.246+0000 7fd4aadc4640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a4102040 0x7fd4a419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.246+0000 7fd4aadc4640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4a4102a00 0x7fd4a419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.246+0000 7fd4aadc4640 1 --2- 172.21.15.132:0/2495819871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4a4100e60 0x7fd4a406b0c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.246+0000 7fd4aadc4640 1 -- 172.21.15.132:0/2495819871 >> 172.21.15.132:0/2495819871 conn(0x7fd4a40fc420 msgr2=0x7fd4a41054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:07.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.246+0000 7fd4aadc4640 1 -- 172.21.15.132:0/2495819871 shutdown_connections 2024-04-23T18:34:07.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.246+0000 7fd4aadc4640 1 -- 172.21.15.132:0/2495819871 wait complete. 2024-04-23T18:34:07.249 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-23T18:34:07.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.360+0000 7fb916708640 1 -- 172.21.15.132:0/804741754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb910101fe0 msgr2=0x7fb910102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.360+0000 7fb916708640 1 --2- 172.21.15.132:0/804741754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb910101fe0 0x7fb910102460 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb904009900 tx=0x7fb90402a410 comp rx=0 tx=0).stop 2024-04-23T18:34:07.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.360+0000 7fb916708640 1 -- 172.21.15.132:0/804741754 shutdown_connections 2024-04-23T18:34:07.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.360+0000 7fb916708640 1 --2- 172.21.15.132:0/804741754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9101029a0 0x7fb9101092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.360+0000 7fb916708640 1 --2- 172.21.15.132:0/804741754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb910101fe0 0x7fb910102460 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.360+0000 7fb916708640 1 --2- 172.21.15.132:0/804741754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb910100e90 0x7fb9101012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.360+0000 7fb916708640 1 -- 172.21.15.132:0/804741754 >> 172.21.15.132:0/804741754 conn(0x7fb9100fc400 msgr2=0x7fb9100fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:07.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.360+0000 7fb916708640 1 -- 172.21.15.132:0/804741754 shutdown_connections 2024-04-23T18:34:07.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.360+0000 7fb916708640 1 -- 172.21.15.132:0/804741754 wait complete. 2024-04-23T18:34:07.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb916708640 1 Processor -- start 2024-04-23T18:34:07.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb916708640 1 -- start start 2024-04-23T18:34:07.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb916708640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb910100e90 0x7fb910198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:07.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb916708640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb910101fe0 0x7fb910198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:07.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb916708640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9101029a0 0x7fb91019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:07.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb916708640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb91010ca40 con 0x7fb910101fe0 2024-04-23T18:34:07.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb916708640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb91010c8c0 con 0x7fb9101029a0 2024-04-23T18:34:07.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb916708640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb91010cbc0 con 0x7fb910100e90 2024-04-23T18:34:07.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb90f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb910101fe0 0x7fb910198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:07.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb90f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb910101fe0 0x7fb910198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36080/0 (socket says 172.21.15.132:36080) 2024-04-23T18:34:07.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb90f7fe640 1 -- 172.21.15.132:0/4253921030 learned_addr learned my addr 172.21.15.132:0/4253921030 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:07.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb914ca5640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9101029a0 0x7fb91019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:07.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.361+0000 7fb90ffff640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb910100e90 0x7fb910198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:07.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.362+0000 7fb90f7fe640 1 -- 172.21.15.132:0/4253921030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb910100e90 msgr2=0x7fb910198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.362+0000 7fb90f7fe640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb910100e90 0x7fb910198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.362+0000 7fb90f7fe640 1 -- 172.21.15.132:0/4253921030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9101029a0 msgr2=0x7fb91019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.362+0000 7fb90f7fe640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9101029a0 0x7fb91019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.362+0000 7fb90f7fe640 1 -- 172.21.15.132:0/4253921030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91019fe60 con 0x7fb910101fe0 2024-04-23T18:34:07.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.362+0000 7fb90f7fe640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb910101fe0 0x7fb910198780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb904031500 tx=0x7fb904003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:07.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.362+0000 7fb90d7fa640 1 -- 172.21.15.132:0/4253921030 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90403f070 con 0x7fb910101fe0 2024-04-23T18:34:07.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.362+0000 7fb90d7fa640 1 -- 172.21.15.132:0/4253921030 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb904031c50 con 0x7fb910101fe0 2024-04-23T18:34:07.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.362+0000 7fb916708640 1 -- 172.21.15.132:0/4253921030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9101a00f0 con 0x7fb910101fe0 2024-04-23T18:34:07.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.362+0000 7fb90d7fa640 1 -- 172.21.15.132:0/4253921030 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904035030 con 0x7fb910101fe0 2024-04-23T18:34:07.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.362+0000 7fb916708640 1 -- 172.21.15.132:0/4253921030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9101a05e0 con 0x7fb910101fe0 2024-04-23T18:34:07.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.363+0000 7fb916708640 1 -- 172.21.15.132:0/4253921030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9101a0840 con 0x7fb910101fe0 2024-04-23T18:34:07.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.364+0000 7fb90d7fa640 1 -- 172.21.15.132:0/4253921030 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fb904042070 con 0x7fb910101fe0 2024-04-23T18:34:07.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.365+0000 7fb90d7fa640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb8f80a3bb0 0x7fb8f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:07.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.365+0000 7fb90ffff640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb8f80a3bb0 0x7fb8f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:07.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.365+0000 7fb90d7fa640 1 -- 172.21.15.132:0/4253921030 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb9040f0a10 con 0x7fb910101fe0 2024-04-23T18:34:07.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.365+0000 7fb90ffff640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb8f80a3bb0 0x7fb8f80a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb900005fd0 tx=0x7fb90000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:07.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.369+0000 7fb90d7fa640 1 -- 172.21.15.132:0/4253921030 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9040bcef0 con 0x7fb910101fe0 2024-04-23T18:34:07.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.582+0000 7fb916708640 1 -- 172.21.15.132:0/4253921030 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb9100697f0 con 0x7fb8f80a3bb0 2024-04-23T18:34:07.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.604+0000 7fb90d7fa640 1 -- 172.21.15.132:0/4253921030 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb9100697f0 con 0x7fb8f80a3bb0 2024-04-23T18:34:07.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.606+0000 7fb916708640 1 -- 172.21.15.132:0/4253921030 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb8f80a3bb0 msgr2=0x7fb8f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.606+0000 7fb916708640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb8f80a3bb0 0x7fb8f80a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb900005fd0 tx=0x7fb90000b390 comp rx=0 tx=0).stop 2024-04-23T18:34:07.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.606+0000 7fb916708640 1 -- 172.21.15.132:0/4253921030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb910101fe0 msgr2=0x7fb910198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.606+0000 7fb916708640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb910101fe0 0x7fb910198780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb904031500 tx=0x7fb904003c10 comp rx=0 tx=0).stop 2024-04-23T18:34:07.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.606+0000 7fb916708640 1 -- 172.21.15.132:0/4253921030 shutdown_connections 2024-04-23T18:34:07.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.606+0000 7fb916708640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb8f80a3bb0 0x7fb8f80a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.606+0000 7fb916708640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9101029a0 0x7fb91019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.606+0000 7fb916708640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb910101fe0 0x7fb910198780 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.606+0000 7fb916708640 1 --2- 172.21.15.132:0/4253921030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb910100e90 0x7fb910198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.606+0000 7fb916708640 1 -- 172.21.15.132:0/4253921030 >> 172.21.15.132:0/4253921030 conn(0x7fb9100fc400 msgr2=0x7fb9100690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:07.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.606+0000 7fb916708640 1 -- 172.21.15.132:0/4253921030 shutdown_connections 2024-04-23T18:34:07.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.606+0000 7fb916708640 1 -- 172.21.15.132:0/4253921030 wait complete. 2024-04-23T18:34:07.609 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-23T18:34:07.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.719+0000 7f4c6a859640 1 -- 172.21.15.132:0/3299668766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c64103190 msgr2=0x7f4c64103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.719+0000 7f4c6a859640 1 --2- 172.21.15.132:0/3299668766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c64103190 0x7f4c64103610 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009900 tx=0x7f4c4c02a410 comp rx=0 tx=0).stop 2024-04-23T18:34:07.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.719+0000 7f4c6a859640 1 -- 172.21.15.132:0/3299668766 shutdown_connections 2024-04-23T18:34:07.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.719+0000 7f4c6a859640 1 --2- 172.21.15.132:0/3299668766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c64103b50 0x7f4c6410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.719+0000 7f4c6a859640 1 --2- 172.21.15.132:0/3299668766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c64103190 0x7f4c64103610 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c6a859640 1 --2- 172.21.15.132:0/3299668766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c64101fa0 0x7f4c641023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c6a859640 1 -- 172.21.15.132:0/3299668766 >> 172.21.15.132:0/3299668766 conn(0x7f4c640fd530 msgr2=0x7f4c640ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:07.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c6a859640 1 -- 172.21.15.132:0/3299668766 shutdown_connections 2024-04-23T18:34:07.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c6a859640 1 -- 172.21.15.132:0/3299668766 wait complete. 2024-04-23T18:34:07.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c6a859640 1 Processor -- start 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c6a859640 1 -- start start 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c6a859640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c64101fa0 0x7f4c64198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c6a859640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c64103190 0x7f4c64198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c6a859640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c64103b50 0x7f4c6419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c6a859640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6410dae0 con 0x7f4c64101fa0 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c6a859640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6410d960 con 0x7f4c64103190 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c6a859640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6410dc60 con 0x7f4c64103b50 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.720+0000 7f4c63fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c64101fa0 0x7f4c64198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c63fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c64101fa0 0x7f4c64198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36098/0 (socket says 172.21.15.132:36098) 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c63fff640 1 -- 172.21.15.132:0/3728542442 learned_addr learned my addr 172.21.15.132:0/3728542442 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c637fe640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c64103190 0x7f4c64198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c68df6640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c64103b50 0x7f4c6419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c63fff640 1 -- 172.21.15.132:0/3728542442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c64103b50 msgr2=0x7f4c6419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c63fff640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c64103b50 0x7f4c6419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c63fff640 1 -- 172.21.15.132:0/3728542442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c64103190 msgr2=0x7f4c64198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c63fff640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c64103190 0x7f4c64198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c63fff640 1 -- 172.21.15.132:0/3728542442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6419fe30 con 0x7f4c64101fa0 2024-04-23T18:34:07.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c63fff640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c64101fa0 0x7f4c64198210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c970 tx=0x7f4c5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:07.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c617fa640 1 -- 172.21.15.132:0/3728542442 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54004220 con 0x7f4c64101fa0 2024-04-23T18:34:07.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c617fa640 1 -- 172.21.15.132:0/3728542442 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c540043c0 con 0x7f4c64101fa0 2024-04-23T18:34:07.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c6a859640 1 -- 172.21.15.132:0/3728542442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c641a0120 con 0x7f4c64101fa0 2024-04-23T18:34:07.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c617fa640 1 -- 172.21.15.132:0/3728542442 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5400f450 con 0x7f4c64101fa0 2024-04-23T18:34:07.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.721+0000 7f4c6a859640 1 -- 172.21.15.132:0/3728542442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c641a0640 con 0x7f4c64101fa0 2024-04-23T18:34:07.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.722+0000 7f4c6a859640 1 -- 172.21.15.132:0/3728542442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c64109480 con 0x7f4c64101fa0 2024-04-23T18:34:07.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.723+0000 7f4c617fa640 1 -- 172.21.15.132:0/3728542442 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4c54003610 con 0x7f4c64101fa0 2024-04-23T18:34:07.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.724+0000 7f4c617fa640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4c500a3bb0 0x7f4c500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:07.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.724+0000 7f4c637fe640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4c500a3bb0 0x7f4c500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:07.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.724+0000 7f4c617fa640 1 -- 172.21.15.132:0/3728542442 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4c540d0080 con 0x7f4c64101fa0 2024-04-23T18:34:07.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.724+0000 7f4c637fe640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4c500a3bb0 0x7f4c500a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c000c00 tx=0x7f4c4c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:07.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.728+0000 7f4c617fa640 1 -- 172.21.15.132:0/3728542442 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c54014030 con 0x7f4c64101fa0 2024-04-23T18:34:07.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:07 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] up:active 2024-04-23T18:34:07.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:07 smithi157 ceph-mon[32580]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-23T18:34:07.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:07 smithi157 ceph-mon[32580]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T18:34:07.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:07 smithi157 ceph-mon[32580]: Cluster is now healthy 2024-04-23T18:34:07.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:07 smithi157 ceph-mon[32580]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-23T18:34:07.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:07 smithi157 ceph-mon[32580]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-23T18:34:07.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:07 smithi157 ceph-mon[32580]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-23T18:34:07.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:07 smithi157 ceph-mon[32580]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:07.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:07 smithi157 ceph-mon[32580]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:07.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:07 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] up:active 2024-04-23T18:34:07.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:07 smithi132 ceph-mon[29178]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-23T18:34:07.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:07 smithi132 ceph-mon[29178]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T18:34:07.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:07 smithi132 ceph-mon[29178]: Cluster is now healthy 2024-04-23T18:34:07.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:07 smithi132 ceph-mon[29178]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-23T18:34:07.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:07 smithi132 ceph-mon[29178]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-23T18:34:07.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:07 smithi132 ceph-mon[29178]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-23T18:34:07.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:07 smithi132 ceph-mon[29178]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:07.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:07 smithi132 ceph-mon[29178]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:07.926 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.934+0000 7f4c6a859640 1 -- 172.21.15.132:0/3728542442 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f4c64062390 con 0x7f4c500a3bb0 2024-04-23T18:34:07.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.939+0000 7f4c617fa640 1 -- 172.21.15.132:0/3728542442 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4c64062390 con 0x7f4c500a3bb0 2024-04-23T18:34:07.931 INFO:teuthology.orchestra.run.smithi132.stdout:/volumes/qa/sv_1/047319ef-c4f2-44a4-baf2-edbcd2a1c750 2024-04-23T18:34:07.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.941+0000 7f4c6a859640 1 -- 172.21.15.132:0/3728542442 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4c500a3bb0 msgr2=0x7f4c500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.941+0000 7f4c6a859640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4c500a3bb0 0x7f4c500a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c000c00 tx=0x7f4c4c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.941+0000 7f4c6a859640 1 -- 172.21.15.132:0/3728542442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c64101fa0 msgr2=0x7f4c64198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:07.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.941+0000 7f4c6a859640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c64101fa0 0x7f4c64198210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c970 tx=0x7f4c5400ce30 comp rx=0 tx=0).stop 2024-04-23T18:34:07.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.941+0000 7f4c6a859640 1 -- 172.21.15.132:0/3728542442 shutdown_connections 2024-04-23T18:34:07.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.941+0000 7f4c6a859640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4c500a3bb0 0x7f4c500a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.941+0000 7f4c6a859640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c64103b50 0x7f4c6419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.941+0000 7f4c6a859640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c64103190 0x7f4c64198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.941+0000 7f4c6a859640 1 --2- 172.21.15.132:0/3728542442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c64101fa0 0x7f4c64198210 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:07.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.941+0000 7f4c6a859640 1 -- 172.21.15.132:0/3728542442 >> 172.21.15.132:0/3728542442 conn(0x7f4c640fd530 msgr2=0x7f4c64106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:07.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.942+0000 7f4c6a859640 1 -- 172.21.15.132:0/3728542442 shutdown_connections 2024-04-23T18:34:07.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:07.942+0000 7f4c6a859640 1 -- 172.21.15.132:0/3728542442 wait complete. 2024-04-23T18:34:07.936 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:07 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] up:active 2024-04-23T18:34:07.936 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:07 smithi148 ceph-mon[32311]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-23T18:34:07.936 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:07 smithi148 ceph-mon[32311]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T18:34:07.936 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:07 smithi148 ceph-mon[32311]: Cluster is now healthy 2024-04-23T18:34:07.936 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:07 smithi148 ceph-mon[32311]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-23T18:34:07.936 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:07 smithi148 ceph-mon[32311]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-23T18:34:07.936 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:07 smithi148 ceph-mon[32311]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-23T18:34:07.936 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:07 smithi148 ceph-mon[32311]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:07.936 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:07 smithi148 ceph-mon[32311]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:07.945 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:34:08.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.052+0000 7fb452b95640 1 -- 172.21.15.132:0/7512849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c101fb0 msgr2=0x7fb44c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.052+0000 7fb452b95640 1 --2- 172.21.15.132:0/7512849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c101fb0 0x7fb44c1023d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb440009900 tx=0x7fb44002a410 comp rx=0 tx=0).stop 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.052+0000 7fb452b95640 1 -- 172.21.15.132:0/7512849 shutdown_connections 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.052+0000 7fb452b95640 1 --2- 172.21.15.132:0/7512849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44c103b60 0x7fb44c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.052+0000 7fb452b95640 1 --2- 172.21.15.132:0/7512849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb44c1031a0 0x7fb44c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.052+0000 7fb452b95640 1 --2- 172.21.15.132:0/7512849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c101fb0 0x7fb44c1023d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.052+0000 7fb452b95640 1 -- 172.21.15.132:0/7512849 >> 172.21.15.132:0/7512849 conn(0x7fb44c0fd500 msgr2=0x7fb44c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.052+0000 7fb452b95640 1 -- 172.21.15.132:0/7512849 shutdown_connections 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.052+0000 7fb452b95640 1 -- 172.21.15.132:0/7512849 wait complete. 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.052+0000 7fb452b95640 1 Processor -- start 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.052+0000 7fb452b95640 1 -- start start 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb452b95640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44c101fb0 0x7fb44c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb452b95640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb44c1031a0 0x7fb44c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb452b95640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c103b60 0x7fb44c19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:08.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb452b95640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c10dbb0 con 0x7fb44c103b60 2024-04-23T18:34:08.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb452b95640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c10da30 con 0x7fb44c1031a0 2024-04-23T18:34:08.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb452b95640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c10dd30 con 0x7fb44c101fb0 2024-04-23T18:34:08.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb451132640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c103b60 0x7fb44c19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:08.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb451132640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c103b60 0x7fb44c19f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36110/0 (socket says 172.21.15.132:36110) 2024-04-23T18:34:08.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb451132640 1 -- 172.21.15.132:0/2736017738 learned_addr learned my addr 172.21.15.132:0/2736017738 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:08.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb44bfff640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb44c1031a0 0x7fb44c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:08.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb450931640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44c101fb0 0x7fb44c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:08.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb451132640 1 -- 172.21.15.132:0/2736017738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44c101fb0 msgr2=0x7fb44c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb451132640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44c101fb0 0x7fb44c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb451132640 1 -- 172.21.15.132:0/2736017738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb44c1031a0 msgr2=0x7fb44c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb451132640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb44c1031a0 0x7fb44c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb451132640 1 -- 172.21.15.132:0/2736017738 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c19ff90 con 0x7fb44c103b60 2024-04-23T18:34:08.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb451132640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c103b60 0x7fb44c19f920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb44400b970 tx=0x7fb44400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:08.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb449ffb640 1 -- 172.21.15.132:0/2736017738 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444019070 con 0x7fb44c103b60 2024-04-23T18:34:08.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb452b95640 1 -- 172.21.15.132:0/2736017738 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44c1a0280 con 0x7fb44c103b60 2024-04-23T18:34:08.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb449ffb640 1 -- 172.21.15.132:0/2736017738 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb444014440 con 0x7fb44c103b60 2024-04-23T18:34:08.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb449ffb640 1 -- 172.21.15.132:0/2736017738 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4440075d0 con 0x7fb44c103b60 2024-04-23T18:34:08.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.053+0000 7fb452b95640 1 -- 172.21.15.132:0/2736017738 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44c1a0850 con 0x7fb44c103b60 2024-04-23T18:34:08.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.054+0000 7fb452b95640 1 -- 172.21.15.132:0/2736017738 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb44c109440 con 0x7fb44c103b60 2024-04-23T18:34:08.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.055+0000 7fb449ffb640 1 -- 172.21.15.132:0/2736017738 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fb44401c070 con 0x7fb44c103b60 2024-04-23T18:34:08.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.056+0000 7fb449ffb640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb43c0a3bb0 0x7fb43c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:08.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.056+0000 7fb450931640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb43c0a3bb0 0x7fb43c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:08.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.056+0000 7fb449ffb640 1 -- 172.21.15.132:0/2736017738 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb4440cf7f0 con 0x7fb44c103b60 2024-04-23T18:34:08.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.056+0000 7fb450931640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb43c0a3bb0 0x7fb43c0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb440000c00 tx=0x7fb4400314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:08.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.060+0000 7fb449ffb640 1 -- 172.21.15.132:0/2736017738 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb44409c320 con 0x7fb44c103b60 2024-04-23T18:34:08.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.279+0000 7fb452b95640 1 -- 172.21.15.132:0/2736017738 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb44c062b50 con 0x7fb44c103b60 2024-04-23T18:34:08.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.280+0000 7fb449ffb640 1 -- 172.21.15.132:0/2736017738 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11946 (secure 0 0 0) 0x7fb44409bc20 con 0x7fb44c103b60 2024-04-23T18:34:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:34:08.272 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:34:07.536766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24394,24425,24460]},"id":1}]} 2024-04-23T18:34:08.272 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 25 2024-04-23T18:34:08.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.282+0000 7fb452b95640 1 -- 172.21.15.132:0/2736017738 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb43c0a3bb0 msgr2=0x7fb43c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.282+0000 7fb452b95640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb43c0a3bb0 0x7fb43c0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb440000c00 tx=0x7fb4400314a0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.282+0000 7fb452b95640 1 -- 172.21.15.132:0/2736017738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c103b60 msgr2=0x7fb44c19f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.282+0000 7fb452b95640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c103b60 0x7fb44c19f920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb44400b970 tx=0x7fb44400bd70 comp rx=0 tx=0).stop 2024-04-23T18:34:08.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.282+0000 7fb452b95640 1 -- 172.21.15.132:0/2736017738 shutdown_connections 2024-04-23T18:34:08.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.282+0000 7fb452b95640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb43c0a3bb0 0x7fb43c0a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.282+0000 7fb452b95640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44c103b60 0x7fb44c19f920 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.282+0000 7fb452b95640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb44c1031a0 0x7fb44c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.282+0000 7fb452b95640 1 --2- 172.21.15.132:0/2736017738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44c101fb0 0x7fb44c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.282+0000 7fb452b95640 1 -- 172.21.15.132:0/2736017738 >> 172.21.15.132:0/2736017738 conn(0x7fb44c0fd500 msgr2=0x7fb44c106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:08.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.282+0000 7fb452b95640 1 -- 172.21.15.132:0/2736017738 shutdown_connections 2024-04-23T18:34:08.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.282+0000 7fb452b95640 1 -- 172.21.15.132:0/2736017738 wait complete. 2024-04-23T18:34:08.286 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:34:08.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.391+0000 7f5c3a413640 1 -- 172.21.15.132:0/3393956798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c34100e90 msgr2=0x7f5c341012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.391+0000 7f5c3a413640 1 --2- 172.21.15.132:0/3393956798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c34100e90 0x7f5c341012b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009900 tx=0x7f5c2402a410 comp rx=0 tx=0).stop 2024-04-23T18:34:08.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.391+0000 7f5c3a413640 1 -- 172.21.15.132:0/3393956798 shutdown_connections 2024-04-23T18:34:08.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.391+0000 7f5c3a413640 1 --2- 172.21.15.132:0/3393956798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c34101fe0 0x7f5c34102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.391+0000 7f5c3a413640 1 --2- 172.21.15.132:0/3393956798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c341029a0 0x7f5c341092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.391+0000 7f5c3a413640 1 --2- 172.21.15.132:0/3393956798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c34100e90 0x7f5c341012b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.391+0000 7f5c3a413640 1 -- 172.21.15.132:0/3393956798 >> 172.21.15.132:0/3393956798 conn(0x7f5c340fc400 msgr2=0x7f5c340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:08.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.392+0000 7f5c3a413640 1 -- 172.21.15.132:0/3393956798 shutdown_connections 2024-04-23T18:34:08.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.392+0000 7f5c3a413640 1 -- 172.21.15.132:0/3393956798 wait complete. 2024-04-23T18:34:08.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.392+0000 7f5c3a413640 1 Processor -- start 2024-04-23T18:34:08.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.392+0000 7f5c3a413640 1 -- start start 2024-04-23T18:34:08.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c3a413640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c34100e90 0x7f5c34198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:08.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c3a413640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c34101fe0 0x7f5c34198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:08.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c3a413640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c341029a0 0x7f5c3419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:08.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c3a413640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3410db30 con 0x7f5c341029a0 2024-04-23T18:34:08.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c3a413640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3410d9b0 con 0x7f5c34100e90 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c3a413640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3410dcb0 con 0x7f5c34101fe0 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c389b0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c341029a0 0x7f5c3419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c389b0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c341029a0 0x7f5c3419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36134/0 (socket says 172.21.15.132:36134) 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c389b0640 1 -- 172.21.15.132:0/1817613016 learned_addr learned my addr 172.21.15.132:0/1817613016 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c33fff640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c34100e90 0x7f5c34198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c337fe640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c34101fe0 0x7f5c34198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c389b0640 1 -- 172.21.15.132:0/1817613016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c34101fe0 msgr2=0x7f5c34198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c389b0640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c34101fe0 0x7f5c34198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c389b0640 1 -- 172.21.15.132:0/1817613016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c34100e90 msgr2=0x7f5c34198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c389b0640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c34100e90 0x7f5c34198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c389b0640 1 -- 172.21.15.132:0/1817613016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3419fe40 con 0x7f5c341029a0 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c389b0640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c341029a0 0x7f5c3419f7d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00eb10 tx=0x7f5c2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c317fa640 1 -- 172.21.15.132:0/1817613016 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c004b20 con 0x7f5c341029a0 2024-04-23T18:34:08.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c3a413640 1 -- 172.21.15.132:0/1817613016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c341a0130 con 0x7f5c341029a0 2024-04-23T18:34:08.389 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c317fa640 1 -- 172.21.15.132:0/1817613016 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c2c004cc0 con 0x7f5c341029a0 2024-04-23T18:34:08.389 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c317fa640 1 -- 172.21.15.132:0/1817613016 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c010670 con 0x7f5c341029a0 2024-04-23T18:34:08.389 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.393+0000 7f5c3a413640 1 -- 172.21.15.132:0/1817613016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c341a0680 con 0x7f5c341029a0 2024-04-23T18:34:08.389 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.394+0000 7f5c3a413640 1 -- 172.21.15.132:0/1817613016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c34108210 con 0x7f5c341029a0 2024-04-23T18:34:08.390 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.396+0000 7f5c317fa640 1 -- 172.21.15.132:0/1817613016 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5c2c005000 con 0x7f5c341029a0 2024-04-23T18:34:08.390 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.396+0000 7f5c317fa640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5c1c0a3bb0 0x7f5c1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:08.390 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.396+0000 7f5c33fff640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5c1c0a3bb0 0x7f5c1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:08.390 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.396+0000 7f5c317fa640 1 -- 172.21.15.132:0/1817613016 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5c2c0d0a40 con 0x7f5c341029a0 2024-04-23T18:34:08.390 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.397+0000 7f5c33fff640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5c1c0a3bb0 0x7f5c1c0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5c24000c00 tx=0x7f5c240314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:08.393 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.400+0000 7f5c317fa640 1 -- 172.21.15.132:0/1817613016 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c2c053ff0 con 0x7f5c341029a0 2024-04-23T18:34:08.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.558+0000 7f5c317fa640 1 -- 172.21.15.132:0/1817613016 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5c2c0994e0 con 0x7f5c341029a0 2024-04-23T18:34:08.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.591+0000 7f5c3a413640 1 -- 172.21.15.132:0/1817613016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c34062b50 con 0x7f5c341029a0 2024-04-23T18:34:08.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.592+0000 7f5c317fa640 1 -- 172.21.15.132:0/1817613016 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22301 (secure 0 0 0) 0x7f5c2c09d570 con 0x7f5c341029a0 2024-04-23T18:34:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:34:08.585 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":86,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:34:02.205993+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:34:08.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.594+0000 7f5c3a413640 1 -- 172.21.15.132:0/1817613016 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5c1c0a3bb0 msgr2=0x7f5c1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.594+0000 7f5c3a413640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5c1c0a3bb0 0x7f5c1c0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5c24000c00 tx=0x7f5c240314a0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.594+0000 7f5c3a413640 1 -- 172.21.15.132:0/1817613016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c341029a0 msgr2=0x7f5c3419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.594+0000 7f5c3a413640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c341029a0 0x7f5c3419f7d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00eb10 tx=0x7f5c2c00c2d0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.594+0000 7f5c3a413640 1 -- 172.21.15.132:0/1817613016 shutdown_connections 2024-04-23T18:34:08.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.594+0000 7f5c3a413640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5c1c0a3bb0 0x7f5c1c0a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.594+0000 7f5c3a413640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c34101fe0 0x7f5c34198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.594+0000 7f5c3a413640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c341029a0 0x7f5c3419f7d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.594+0000 7f5c3a413640 1 --2- 172.21.15.132:0/1817613016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c34100e90 0x7f5c34198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.594+0000 7f5c3a413640 1 -- 172.21.15.132:0/1817613016 >> 172.21.15.132:0/1817613016 conn(0x7f5c340fc400 msgr2=0x7f5c341053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:08.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.594+0000 7f5c3a413640 1 -- 172.21.15.132:0/1817613016 shutdown_connections 2024-04-23T18:34:08.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.594+0000 7f5c3a413640 1 -- 172.21.15.132:0/1817613016 wait complete. 2024-04-23T18:34:08.596 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:08 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] up:active 2024-04-23T18:34:08.597 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:08 smithi132 ceph-mon[29178]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-23T18:34:08.597 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:08 smithi132 ceph-mon[29178]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:34:08.597 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:08 smithi132 ceph-mon[29178]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:08.597 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:08 smithi132 ceph-mon[29178]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:08.597 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:08 smithi132 ceph-mon[29178]: pgmap v293: 97 pgs: 97 active+clean; 583 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 5.2 KiB/s wr, 18 op/s 2024-04-23T18:34:08.597 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:08 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2736017738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:08.598 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T18:34:08.599 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:34:08.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.704+0000 7f34d9f43640 1 -- 172.21.15.132:0/628992161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4105350 msgr2=0x7f34d4107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.704+0000 7f34d9f43640 1 --2- 172.21.15.132:0/628992161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4105350 0x7f34d4107740 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009900 tx=0x7f34c402a410 comp rx=0 tx=0).stop 2024-04-23T18:34:08.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.704+0000 7f34d9f43640 1 -- 172.21.15.132:0/628992161 shutdown_connections 2024-04-23T18:34:08.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.704+0000 7f34d9f43640 1 --2- 172.21.15.132:0/628992161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34d4107c80 0x7f34d410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.704+0000 7f34d9f43640 1 --2- 172.21.15.132:0/628992161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4105350 0x7f34d4107740 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.704+0000 7f34d9f43640 1 --2- 172.21.15.132:0/628992161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34d4102a20 0x7f34d4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.704+0000 7f34d9f43640 1 -- 172.21.15.132:0/628992161 >> 172.21.15.132:0/628992161 conn(0x7f34d40fc430 msgr2=0x7f34d40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:08.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.704+0000 7f34d9f43640 1 -- 172.21.15.132:0/628992161 shutdown_connections 2024-04-23T18:34:08.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.704+0000 7f34d9f43640 1 -- 172.21.15.132:0/628992161 wait complete. 2024-04-23T18:34:08.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.704+0000 7f34d9f43640 1 Processor -- start 2024-04-23T18:34:08.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.704+0000 7f34d9f43640 1 -- start start 2024-04-23T18:34:08.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d9f43640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34d4102a20 0x7f34d4198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:08.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d9f43640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4105350 0x7f34d4198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:08.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d9f43640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34d4107c80 0x7f34d419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:08.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d9f43640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f34d410db20 con 0x7f34d4105350 2024-04-23T18:34:08.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d9f43640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f34d410d9a0 con 0x7f34d4107c80 2024-04-23T18:34:08.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d9f43640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34d410dca0 con 0x7f34d4102a20 2024-04-23T18:34:08.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d2ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4105350 0x7f34d4198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d2ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4105350 0x7f34d4198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36148/0 (socket says 172.21.15.132:36148) 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d2ffd640 1 -- 172.21.15.132:0/4207660428 learned_addr learned my addr 172.21.15.132:0/4207660428 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d3fff640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34d4107c80 0x7f34d419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d2ffd640 1 -- 172.21.15.132:0/4207660428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34d4102a20 msgr2=0x7f34d4198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d2ffd640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34d4102a20 0x7f34d4198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d2ffd640 1 -- 172.21.15.132:0/4207660428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34d4107c80 msgr2=0x7f34d419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d2ffd640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34d4107c80 0x7f34d419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d2ffd640 1 -- 172.21.15.132:0/4207660428 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d419fe70 con 0x7f34d4105350 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d2ffd640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4105350 0x7f34d4198790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f34c4006010 tx=0x7f34c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d0ff9640 1 -- 172.21.15.132:0/4207660428 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c403f030 con 0x7f34d4105350 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d9f43640 1 -- 172.21.15.132:0/4207660428 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d41a0100 con 0x7f34d4105350 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d0ff9640 1 -- 172.21.15.132:0/4207660428 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c40318f0 con 0x7f34d4105350 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d0ff9640 1 -- 172.21.15.132:0/4207660428 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4035070 con 0x7f34d4105350 2024-04-23T18:34:08.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.705+0000 7f34d9f43640 1 -- 172.21.15.132:0/4207660428 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d41a06c0 con 0x7f34d4105350 2024-04-23T18:34:08.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.706+0000 7f34d9f43640 1 -- 172.21.15.132:0/4207660428 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d40fdf20 con 0x7f34d4105350 2024-04-23T18:34:08.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.707+0000 7f34d0ff9640 1 -- 172.21.15.132:0/4207660428 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f34c4042030 con 0x7f34d4105350 2024-04-23T18:34:08.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.708+0000 7f34d0ff9640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f34bc0a3c40 0x7f34bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:08.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.708+0000 7f34d37fe640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f34bc0a3c40 0x7f34bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:08.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.708+0000 7f34d0ff9640 1 -- 172.21.15.132:0/4207660428 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f34c40f0d30 con 0x7f34d4105350 2024-04-23T18:34:08.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.708+0000 7f34d37fe640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f34bc0a3c40 0x7f34bc0a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f34c80058c0 tx=0x7f34c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:08.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.712+0000 7f34d0ff9640 1 -- 172.21.15.132:0/4207660428 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34c40bd760 con 0x7f34d4105350 2024-04-23T18:34:08.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:08 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] up:active 2024-04-23T18:34:08.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:08 smithi157 ceph-mon[32580]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-23T18:34:08.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:08 smithi157 ceph-mon[32580]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:34:08.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:08 smithi157 ceph-mon[32580]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:08.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:08 smithi157 ceph-mon[32580]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:08.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:08 smithi157 ceph-mon[32580]: pgmap v293: 97 pgs: 97 active+clean; 583 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 5.2 KiB/s wr, 18 op/s 2024-04-23T18:34:08.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:08 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2736017738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:08.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.889+0000 7f34d9f43640 1 -- 172.21.15.132:0/4207660428 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f34d4062390 con 0x7f34bc0a3c40 2024-04-23T18:34:08.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.897+0000 7f34d0ff9640 1 -- 172.21.15.132:0/4207660428 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258947 (secure 0 0 0) 0x7f34d4062390 con 0x7f34bc0a3c40 2024-04-23T18:34:08.892 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:34:08.894 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-04-23T18:34:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.900+0000 7f34d9f43640 1 -- 172.21.15.132:0/4207660428 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f34bc0a3c40 msgr2=0x7f34bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.900+0000 7f34d9f43640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f34bc0a3c40 0x7f34bc0a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f34c80058c0 tx=0x7f34c8005850 comp rx=0 tx=0).stop 2024-04-23T18:34:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.900+0000 7f34d9f43640 1 -- 172.21.15.132:0/4207660428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4105350 msgr2=0x7f34d4198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.900+0000 7f34d9f43640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4105350 0x7f34d4198790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f34c4006010 tx=0x7f34c40042f0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.900+0000 7f34d9f43640 1 -- 172.21.15.132:0/4207660428 shutdown_connections 2024-04-23T18:34:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.900+0000 7f34d9f43640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f34bc0a3c40 0x7f34bc0a6100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.900+0000 7f34d9f43640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34d4107c80 0x7f34d419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.900+0000 7f34d9f43640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4105350 0x7f34d4198790 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.900+0000 7f34d9f43640 1 --2- 172.21.15.132:0/4207660428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34d4102a20 0x7f34d4198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.900+0000 7f34d9f43640 1 -- 172.21.15.132:0/4207660428 >> 172.21.15.132:0/4207660428 conn(0x7f34d40fc430 msgr2=0x7f34d41034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.900+0000 7f34d9f43640 1 -- 172.21.15.132:0/4207660428 shutdown_connections 2024-04-23T18:34:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:08.900+0000 7f34d9f43640 1 -- 172.21.15.132:0/4207660428 wait complete. 2024-04-23T18:34:08.897 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:08 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] up:active 2024-04-23T18:34:08.897 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:08 smithi148 ceph-mon[32311]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-23T18:34:08.897 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:08 smithi148 ceph-mon[32311]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:34:08.897 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:08 smithi148 ceph-mon[32311]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:08.897 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:08 smithi148 ceph-mon[32311]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:08.897 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:08 smithi148 ceph-mon[32311]: pgmap v293: 97 pgs: 97 active+clean; 583 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 5.2 KiB/s wr, 18 op/s 2024-04-23T18:34:08.897 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:08 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2736017738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:08.904 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-04-23T18:34:08.125447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596850,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":144,"num_read_kb":127,"num_write":322,"num_write_kb":2979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":260,"ondisk_log_size":260,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352116,"kb_used_data":11256,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589708,"statfs":{"total":1151940427776,"available":1151579860992,"internally_reserved":0,"allocated":11526144,"data_stored":7116715,"data_compressed":107658,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6482,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":141,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.915297"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338134+0000","last_change":"2024-04-23T18:34:01.231204+0000","last_active":"2024-04-23T18:34:02.338134+0000","last_peered":"2024-04-23T18:34:02.338134+0000","last_clean":"2024-04-23T18:34:02.338134+0000","last_became_active":"2024-04-23T18:34:01.230836+0000","last_became_peered":"2024-04-23T18:34:01.230836+0000","last_unstale":"2024-04-23T18:34:02.338134+0000","last_undegraded":"2024-04-23T18:34:02.338134+0000","last_fullsized":"2024-04-23T18:34:02.338134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:57.744807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338435+0000","last_change":"2024-04-23T18:34:01.225833+0000","last_active":"2024-04-23T18:34:02.338435+0000","last_peered":"2024-04-23T18:34:02.338435+0000","last_clean":"2024-04-23T18:34:02.338435+0000","last_became_active":"2024-04-23T18:34:01.225098+0000","last_became_peered":"2024-04-23T18:34:01.225098+0000","last_unstale":"2024-04-23T18:34:02.338435+0000","last_undegraded":"2024-04-23T18:34:02.338435+0000","last_fullsized":"2024-04-23T18:34:02.338435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:20.720137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338461+0000","last_change":"2024-04-23T18:34:01.235269+0000","last_active":"2024-04-23T18:34:02.338461+0000","last_peered":"2024-04-23T18:34:02.338461+0000","last_clean":"2024-04-23T18:34:02.338461+0000","last_became_active":"2024-04-23T18:34:01.234921+0000","last_became_peered":"2024-04-23T18:34:01.234921+0000","last_unstale":"2024-04-23T18:34:02.338461+0000","last_undegraded":"2024-04-23T18:34:02.338461+0000","last_fullsized":"2024-04-23T18:34:02.338461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:06:20.645693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214493+0000","last_change":"2024-04-23T18:34:01.233316+0000","last_active":"2024-04-23T18:34:02.214493+0000","last_peered":"2024-04-23T18:34:02.214493+0000","last_clean":"2024-04-23T18:34:02.214493+0000","last_became_active":"2024-04-23T18:34:01.231636+0000","last_became_peered":"2024-04-23T18:34:01.231636+0000","last_unstale":"2024-04-23T18:34:02.214493+0000","last_undegraded":"2024-04-23T18:34:02.214493+0000","last_fullsized":"2024-04-23T18:34:02.214493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:15.837129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.839838+0000","last_change":"2024-04-23T18:34:01.228073+0000","last_active":"2024-04-23T18:34:02.839838+0000","last_peered":"2024-04-23T18:34:02.839838+0000","last_clean":"2024-04-23T18:34:02.839838+0000","last_became_active":"2024-04-23T18:34:01.227701+0000","last_became_peered":"2024-04-23T18:34:01.227701+0000","last_unstale":"2024-04-23T18:34:02.839838+0000","last_undegraded":"2024-04-23T18:34:02.839838+0000","last_fullsized":"2024-04-23T18:34:02.839838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:17:26.556498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338952+0000","last_change":"2024-04-23T18:34:01.228033+0000","last_active":"2024-04-23T18:34:02.338952+0000","last_peered":"2024-04-23T18:34:02.338952+0000","last_clean":"2024-04-23T18:34:02.338952+0000","last_became_active":"2024-04-23T18:34:01.227665+0000","last_became_peered":"2024-04-23T18:34:01.227665+0000","last_unstale":"2024-04-23T18:34:02.338952+0000","last_undegraded":"2024-04-23T18:34:02.338952+0000","last_fullsized":"2024-04-23T18:34:02.338952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:53:07.663983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337775+0000","last_change":"2024-04-23T18:34:01.238484+0000","last_active":"2024-04-23T18:34:02.337775+0000","last_peered":"2024-04-23T18:34:02.337775+0000","last_clean":"2024-04-23T18:34:02.337775+0000","last_became_active":"2024-04-23T18:34:01.238091+0000","last_became_peered":"2024-04-23T18:34:01.238091+0000","last_unstale":"2024-04-23T18:34:02.337775+0000","last_undegraded":"2024-04-23T18:34:02.337775+0000","last_fullsized":"2024-04-23T18:34:02.337775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:52.472116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213438+0000","last_change":"2024-04-23T18:34:01.233934+0000","last_active":"2024-04-23T18:34:02.213438+0000","last_peered":"2024-04-23T18:34:02.213438+0000","last_clean":"2024-04-23T18:34:02.213438+0000","last_became_active":"2024-04-23T18:34:01.233597+0000","last_became_peered":"2024-04-23T18:34:01.233597+0000","last_unstale":"2024-04-23T18:34:02.213438+0000","last_undegraded":"2024-04-23T18:34:02.213438+0000","last_fullsized":"2024-04-23T18:34:02.213438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:33:51.911420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338587+0000","last_change":"2024-04-23T18:34:01.234094+0000","last_active":"2024-04-23T18:34:02.338587+0000","last_peered":"2024-04-23T18:34:02.338587+0000","last_clean":"2024-04-23T18:34:02.338587+0000","last_became_active":"2024-04-23T18:34:01.232499+0000","last_became_peered":"2024-04-23T18:34:01.232499+0000","last_unstale":"2024-04-23T18:34:02.338587+0000","last_undegraded":"2024-04-23T18:34:02.338587+0000","last_fullsized":"2024-04-23T18:34:02.338587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:33.004245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.339353+0000","last_change":"2024-04-23T18:34:01.229175+0000","last_active":"2024-04-23T18:34:02.339353+0000","last_peered":"2024-04-23T18:34:02.339353+0000","last_clean":"2024-04-23T18:34:02.339353+0000","last_became_active":"2024-04-23T18:34:01.228837+0000","last_became_peered":"2024-04-23T18:34:01.228837+0000","last_unstale":"2024-04-23T18:34:02.339353+0000","last_undegraded":"2024-04-23T18:34:02.339353+0000","last_fullsized":"2024-04-23T18:34:02.339353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:57.625110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338125+0000","last_change":"2024-04-23T18:34:01.235557+0000","last_active":"2024-04-23T18:34:02.338125+0000","last_peered":"2024-04-23T18:34:02.338125+0000","last_clean":"2024-04-23T18:34:02.338125+0000","last_became_active":"2024-04-23T18:34:01.235184+0000","last_became_peered":"2024-04-23T18:34:01.235184+0000","last_unstale":"2024-04-23T18:34:02.338125+0000","last_undegraded":"2024-04-23T18:34:02.338125+0000","last_fullsized":"2024-04-23T18:34:02.338125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:37:27.138041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214421+0000","last_change":"2024-04-23T18:34:01.234070+0000","last_active":"2024-04-23T18:34:02.214421+0000","last_peered":"2024-04-23T18:34:02.214421+0000","last_clean":"2024-04-23T18:34:02.214421+0000","last_became_active":"2024-04-23T18:34:01.233727+0000","last_became_peered":"2024-04-23T18:34:01.233727+0000","last_unstale":"2024-04-23T18:34:02.214421+0000","last_undegraded":"2024-04-23T18:34:02.214421+0000","last_fullsized":"2024-04-23T18:34:02.214421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:59:16.489819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338473+0000","last_change":"2024-04-23T18:34:01.228256+0000","last_active":"2024-04-23T18:34:02.338473+0000","last_peered":"2024-04-23T18:34:02.338473+0000","last_clean":"2024-04-23T18:34:02.338473+0000","last_became_active":"2024-04-23T18:34:01.224400+0000","last_became_peered":"2024-04-23T18:34:01.224400+0000","last_unstale":"2024-04-23T18:34:02.338473+0000","last_undegraded":"2024-04-23T18:34:02.338473+0000","last_fullsized":"2024-04-23T18:34:02.338473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:37:28.401695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337550+0000","last_change":"2024-04-23T18:34:01.227250+0000","last_active":"2024-04-23T18:34:02.337550+0000","last_peered":"2024-04-23T18:34:02.337550+0000","last_clean":"2024-04-23T18:34:02.337550+0000","last_became_active":"2024-04-23T18:34:01.226937+0000","last_became_peered":"2024-04-23T18:34:01.226937+0000","last_unstale":"2024-04-23T18:34:02.337550+0000","last_undegraded":"2024-04-23T18:34:02.337550+0000","last_fullsized":"2024-04-23T18:34:02.337550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:26:20.801691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337432+0000","last_change":"2024-04-23T18:34:01.232351+0000","last_active":"2024-04-23T18:34:02.337432+0000","last_peered":"2024-04-23T18:34:02.337432+0000","last_clean":"2024-04-23T18:34:02.337432+0000","last_became_active":"2024-04-23T18:34:01.231987+0000","last_became_peered":"2024-04-23T18:34:01.231987+0000","last_unstale":"2024-04-23T18:34:02.337432+0000","last_undegraded":"2024-04-23T18:34:02.337432+0000","last_fullsized":"2024-04-23T18:34:02.337432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:50.886496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337521+0000","last_change":"2024-04-23T18:34:01.232196+0000","last_active":"2024-04-23T18:34:02.337521+0000","last_peered":"2024-04-23T18:34:02.337521+0000","last_clean":"2024-04-23T18:34:02.337521+0000","last_became_active":"2024-04-23T18:34:01.231746+0000","last_became_peered":"2024-04-23T18:34:01.231746+0000","last_unstale":"2024-04-23T18:34:02.337521+0000","last_undegraded":"2024-04-23T18:34:02.337521+0000","last_fullsized":"2024-04-23T18:34:02.337521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:58.888549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338302+0000","last_change":"2024-04-23T18:34:01.231509+0000","last_active":"2024-04-23T18:34:02.338302+0000","last_peered":"2024-04-23T18:34:02.338302+0000","last_clean":"2024-04-23T18:34:02.338302+0000","last_became_active":"2024-04-23T18:34:01.231017+0000","last_became_peered":"2024-04-23T18:34:01.231017+0000","last_unstale":"2024-04-23T18:34:02.338302+0000","last_undegraded":"2024-04-23T18:34:02.338302+0000","last_fullsized":"2024-04-23T18:34:02.338302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:04:51.929462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338089+0000","last_change":"2024-04-23T18:34:01.223285+0000","last_active":"2024-04-23T18:34:02.338089+0000","last_peered":"2024-04-23T18:34:02.338089+0000","last_clean":"2024-04-23T18:34:02.338089+0000","last_became_active":"2024-04-23T18:34:01.222962+0000","last_became_peered":"2024-04-23T18:34:01.222962+0000","last_unstale":"2024-04-23T18:34:02.338089+0000","last_undegraded":"2024-04-23T18:34:02.338089+0000","last_fullsized":"2024-04-23T18:34:02.338089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:18.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338026+0000","last_change":"2024-04-23T18:34:01.225922+0000","last_active":"2024-04-23T18:34:02.338026+0000","last_peered":"2024-04-23T18:34:02.338026+0000","last_clean":"2024-04-23T18:34:02.338026+0000","last_became_active":"2024-04-23T18:34:01.225235+0000","last_became_peered":"2024-04-23T18:34:01.225235+0000","last_unstale":"2024-04-23T18:34:02.338026+0000","last_undegraded":"2024-04-23T18:34:02.338026+0000","last_fullsized":"2024-04-23T18:34:02.338026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:33.480782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337427+0000","last_change":"2024-04-23T18:34:00.224606+0000","last_active":"2024-04-23T18:34:02.337427+0000","last_peered":"2024-04-23T18:34:02.337427+0000","last_clean":"2024-04-23T18:34:02.337427+0000","last_became_active":"2024-04-23T18:34:00.224291+0000","last_became_peered":"2024-04-23T18:34:00.224291+0000","last_unstale":"2024-04-23T18:34:02.337427+0000","last_undegraded":"2024-04-23T18:34:02.337427+0000","last_fullsized":"2024-04-23T18:34:02.337427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:40.734704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213549+0000","last_change":"2024-04-23T18:34:01.235023+0000","last_active":"2024-04-23T18:34:02.213549+0000","last_peered":"2024-04-23T18:34:02.213549+0000","last_clean":"2024-04-23T18:34:02.213549+0000","last_became_active":"2024-04-23T18:34:01.234672+0000","last_became_peered":"2024-04-23T18:34:01.234672+0000","last_unstale":"2024-04-23T18:34:02.213549+0000","last_undegraded":"2024-04-23T18:34:02.213549+0000","last_fullsized":"2024-04-23T18:34:02.213549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:33.734343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213943+0000","last_change":"2024-04-23T18:34:00.232552+0000","last_active":"2024-04-23T18:34:02.213943+0000","last_peered":"2024-04-23T18:34:02.213943+0000","last_clean":"2024-04-23T18:34:02.213943+0000","last_became_active":"2024-04-23T18:34:00.232226+0000","last_became_peered":"2024-04-23T18:34:00.232226+0000","last_unstale":"2024-04-23T18:34:02.213943+0000","last_undegraded":"2024-04-23T18:34:02.213943+0000","last_fullsized":"2024-04-23T18:34:02.213943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.563757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338458+0000","last_change":"2024-04-23T18:34:01.236013+0000","last_active":"2024-04-23T18:34:02.338458+0000","last_peered":"2024-04-23T18:34:02.338458+0000","last_clean":"2024-04-23T18:34:02.338458+0000","last_became_active":"2024-04-23T18:34:01.235590+0000","last_became_peered":"2024-04-23T18:34:01.235590+0000","last_unstale":"2024-04-23T18:34:02.338458+0000","last_undegraded":"2024-04-23T18:34:02.338458+0000","last_fullsized":"2024-04-23T18:34:02.338458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:45.741648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:07.539158+0000","last_change":"2024-04-23T18:34:00.232336+0000","last_active":"2024-04-23T18:34:07.539158+0000","last_peered":"2024-04-23T18:34:07.539158+0000","last_clean":"2024-04-23T18:34:07.539158+0000","last_became_active":"2024-04-23T18:34:00.231839+0000","last_became_peered":"2024-04-23T18:34:00.231839+0000","last_unstale":"2024-04-23T18:34:07.539158+0000","last_undegraded":"2024-04-23T18:34:07.539158+0000","last_fullsized":"2024-04-23T18:34:07.539158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:38.029434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338067+0000","last_change":"2024-04-23T18:34:01.230435+0000","last_active":"2024-04-23T18:34:02.338067+0000","last_peered":"2024-04-23T18:34:02.338067+0000","last_clean":"2024-04-23T18:34:02.338067+0000","last_became_active":"2024-04-23T18:34:01.230095+0000","last_became_peered":"2024-04-23T18:34:01.230095+0000","last_unstale":"2024-04-23T18:34:02.338067+0000","last_undegraded":"2024-04-23T18:34:02.338067+0000","last_fullsized":"2024-04-23T18:34:02.338067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:31.653254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:06.545234+0000","last_change":"2024-04-23T18:34:00.227853+0000","last_active":"2024-04-23T18:34:06.545234+0000","last_peered":"2024-04-23T18:34:06.545234+0000","last_clean":"2024-04-23T18:34:06.545234+0000","last_became_active":"2024-04-23T18:34:00.226596+0000","last_became_peered":"2024-04-23T18:34:00.226596+0000","last_unstale":"2024-04-23T18:34:06.545234+0000","last_undegraded":"2024-04-23T18:34:06.545234+0000","last_fullsized":"2024-04-23T18:34:06.545234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:16:10.077969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214447+0000","last_change":"2024-04-23T18:34:01.235955+0000","last_active":"2024-04-23T18:34:02.214447+0000","last_peered":"2024-04-23T18:34:02.214447+0000","last_clean":"2024-04-23T18:34:02.214447+0000","last_became_active":"2024-04-23T18:34:01.235609+0000","last_became_peered":"2024-04-23T18:34:01.235609+0000","last_unstale":"2024-04-23T18:34:02.214447+0000","last_undegraded":"2024-04-23T18:34:02.214447+0000","last_fullsized":"2024-04-23T18:34:02.214447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:07:39.480213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.541425+0000","last_change":"2024-04-23T18:34:00.226319+0000","last_active":"2024-04-23T18:34:03.541425+0000","last_peered":"2024-04-23T18:34:03.541425+0000","last_clean":"2024-04-23T18:34:03.541425+0000","last_became_active":"2024-04-23T18:34:00.225796+0000","last_became_peered":"2024-04-23T18:34:00.225796+0000","last_unstale":"2024-04-23T18:34:03.541425+0000","last_undegraded":"2024-04-23T18:34:03.541425+0000","last_fullsized":"2024-04-23T18:34:03.541425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:11:18.309181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.839858+0000","last_change":"2024-04-23T18:34:01.227336+0000","last_active":"2024-04-23T18:34:02.839858+0000","last_peered":"2024-04-23T18:34:02.839858+0000","last_clean":"2024-04-23T18:34:02.839858+0000","last_became_active":"2024-04-23T18:34:01.226987+0000","last_became_peered":"2024-04-23T18:34:01.226987+0000","last_unstale":"2024-04-23T18:34:02.839858+0000","last_undegraded":"2024-04-23T18:34:02.839858+0000","last_fullsized":"2024-04-23T18:34:02.839858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:22:30.058673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.548594+0000","last_change":"2024-04-23T18:34:00.222748+0000","last_active":"2024-04-23T18:34:04.548594+0000","last_peered":"2024-04-23T18:34:04.548594+0000","last_clean":"2024-04-23T18:34:04.548594+0000","last_became_active":"2024-04-23T18:34:00.222412+0000","last_became_peered":"2024-04-23T18:34:00.222412+0000","last_unstale":"2024-04-23T18:34:04.548594+0000","last_undegraded":"2024-04-23T18:34:04.548594+0000","last_fullsized":"2024-04-23T18:34:04.548594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:48:24.807152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214270+0000","last_change":"2024-04-23T18:34:01.234354+0000","last_active":"2024-04-23T18:34:02.214270+0000","last_peered":"2024-04-23T18:34:02.214270+0000","last_clean":"2024-04-23T18:34:02.214270+0000","last_became_active":"2024-04-23T18:34:01.233815+0000","last_became_peered":"2024-04-23T18:34:01.233815+0000","last_unstale":"2024-04-23T18:34:02.214270+0000","last_undegraded":"2024-04-23T18:34:02.214270+0000","last_fullsized":"2024-04-23T18:34:02.214270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:58.793701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.550170+0000","last_change":"2024-04-23T18:34:00.227595+0000","last_active":"2024-04-23T18:34:04.550170+0000","last_peered":"2024-04-23T18:34:04.550170+0000","last_clean":"2024-04-23T18:34:04.550170+0000","last_became_active":"2024-04-23T18:34:00.226830+0000","last_became_peered":"2024-04-23T18:34:00.226830+0000","last_unstale":"2024-04-23T18:34:04.550170+0000","last_undegraded":"2024-04-23T18:34:04.550170+0000","last_fullsized":"2024-04-23T18:34:04.550170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:20.661984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213674+0000","last_change":"2024-04-23T18:34:01.225053+0000","last_active":"2024-04-23T18:34:02.213674+0000","last_peered":"2024-04-23T18:34:02.213674+0000","last_clean":"2024-04-23T18:34:02.213674+0000","last_became_active":"2024-04-23T18:34:01.224723+0000","last_became_peered":"2024-04-23T18:34:01.224723+0000","last_unstale":"2024-04-23T18:34:02.213674+0000","last_undegraded":"2024-04-23T18:34:02.213674+0000","last_fullsized":"2024-04-23T18:34:02.213674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:01:51.486691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.537458+0000","last_change":"2024-04-23T18:34:00.224783+0000","last_active":"2024-04-23T18:34:03.537458+0000","last_peered":"2024-04-23T18:34:03.537458+0000","last_clean":"2024-04-23T18:34:03.537458+0000","last_became_active":"2024-04-23T18:34:00.224436+0000","last_became_peered":"2024-04-23T18:34:00.224436+0000","last_unstale":"2024-04-23T18:34:03.537458+0000","last_undegraded":"2024-04-23T18:34:03.537458+0000","last_fullsized":"2024-04-23T18:34:03.537458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:33.829188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337541+0000","last_change":"2024-04-23T18:34:01.224283+0000","last_active":"2024-04-23T18:34:02.337541+0000","last_peered":"2024-04-23T18:34:02.337541+0000","last_clean":"2024-04-23T18:34:02.337541+0000","last_became_active":"2024-04-23T18:34:01.223949+0000","last_became_peered":"2024-04-23T18:34:01.223949+0000","last_unstale":"2024-04-23T18:34:02.337541+0000","last_undegraded":"2024-04-23T18:34:02.337541+0000","last_fullsized":"2024-04-23T18:34:02.337541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:50.560687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.539367+0000","last_change":"2024-04-23T18:34:00.218453+0000","last_active":"2024-04-23T18:34:03.539367+0000","last_peered":"2024-04-23T18:34:03.539367+0000","last_clean":"2024-04-23T18:34:03.539367+0000","last_became_active":"2024-04-23T18:34:00.218036+0000","last_became_peered":"2024-04-23T18:34:00.218036+0000","last_unstale":"2024-04-23T18:34:03.539367+0000","last_undegraded":"2024-04-23T18:34:03.539367+0000","last_fullsized":"2024-04-23T18:34:03.539367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:49.485185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338427+0000","last_change":"2024-04-23T18:34:01.231143+0000","last_active":"2024-04-23T18:34:02.338427+0000","last_peered":"2024-04-23T18:34:02.338427+0000","last_clean":"2024-04-23T18:34:02.338427+0000","last_became_active":"2024-04-23T18:34:01.230812+0000","last_became_peered":"2024-04-23T18:34:01.230812+0000","last_unstale":"2024-04-23T18:34:02.338427+0000","last_undegraded":"2024-04-23T18:34:02.338427+0000","last_fullsized":"2024-04-23T18:34:02.338427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:28.704807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:06.544251+0000","last_change":"2024-04-23T18:34:00.235711+0000","last_active":"2024-04-23T18:34:06.544251+0000","last_peered":"2024-04-23T18:34:06.544251+0000","last_clean":"2024-04-23T18:34:06.544251+0000","last_became_active":"2024-04-23T18:34:00.235333+0000","last_became_peered":"2024-04-23T18:34:00.235333+0000","last_unstale":"2024-04-23T18:34:06.544251+0000","last_undegraded":"2024-04-23T18:34:06.544251+0000","last_fullsized":"2024-04-23T18:34:06.544251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:51.320744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338458+0000","last_change":"2024-04-23T18:34:01.238291+0000","last_active":"2024-04-23T18:34:02.338458+0000","last_peered":"2024-04-23T18:34:02.338458+0000","last_clean":"2024-04-23T18:34:02.338458+0000","last_became_active":"2024-04-23T18:34:01.237972+0000","last_became_peered":"2024-04-23T18:34:01.237972+0000","last_unstale":"2024-04-23T18:34:02.338458+0000","last_undegraded":"2024-04-23T18:34:02.338458+0000","last_fullsized":"2024-04-23T18:34:02.338458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:36:17.108170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.538485+0000","last_change":"2024-04-23T18:34:00.227249+0000","last_active":"2024-04-23T18:34:03.538485+0000","last_peered":"2024-04-23T18:34:03.538485+0000","last_clean":"2024-04-23T18:34:03.538485+0000","last_became_active":"2024-04-23T18:34:00.226854+0000","last_became_peered":"2024-04-23T18:34:00.226854+0000","last_unstale":"2024-04-23T18:34:03.538485+0000","last_undegraded":"2024-04-23T18:34:03.538485+0000","last_fullsized":"2024-04-23T18:34:03.538485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:49.619906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338126+0000","last_change":"2024-04-23T18:34:01.225518+0000","last_active":"2024-04-23T18:34:02.338126+0000","last_peered":"2024-04-23T18:34:02.338126+0000","last_clean":"2024-04-23T18:34:02.338126+0000","last_became_active":"2024-04-23T18:34:01.224907+0000","last_became_peered":"2024-04-23T18:34:01.224907+0000","last_unstale":"2024-04-23T18:34:02.338126+0000","last_undegraded":"2024-04-23T18:34:02.338126+0000","last_fullsized":"2024-04-23T18:34:02.338126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:19.876629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.538881+0000","last_change":"2024-04-23T18:34:00.224770+0000","last_active":"2024-04-23T18:34:03.538881+0000","last_peered":"2024-04-23T18:34:03.538881+0000","last_clean":"2024-04-23T18:34:03.538881+0000","last_became_active":"2024-04-23T18:34:00.224395+0000","last_became_peered":"2024-04-23T18:34:00.224395+0000","last_unstale":"2024-04-23T18:34:03.538881+0000","last_undegraded":"2024-04-23T18:34:03.538881+0000","last_fullsized":"2024-04-23T18:34:03.538881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:14:09.172479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.840122+0000","last_change":"2024-04-23T18:34:01.227771+0000","last_active":"2024-04-23T18:34:02.840122+0000","last_peered":"2024-04-23T18:34:02.840122+0000","last_clean":"2024-04-23T18:34:02.840122+0000","last_became_active":"2024-04-23T18:34:01.225063+0000","last_became_peered":"2024-04-23T18:34:01.225063+0000","last_unstale":"2024-04-23T18:34:02.840122+0000","last_undegraded":"2024-04-23T18:34:02.840122+0000","last_fullsized":"2024-04-23T18:34:02.840122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:53.921587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.547953+0000","last_change":"2024-04-23T18:34:00.230455+0000","last_active":"2024-04-23T18:34:04.547953+0000","last_peered":"2024-04-23T18:34:04.547953+0000","last_clean":"2024-04-23T18:34:04.547953+0000","last_became_active":"2024-04-23T18:34:00.229934+0000","last_became_peered":"2024-04-23T18:34:00.229934+0000","last_unstale":"2024-04-23T18:34:04.547953+0000","last_undegraded":"2024-04-23T18:34:04.547953+0000","last_fullsized":"2024-04-23T18:34:04.547953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:26:34.787606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337787+0000","last_change":"2024-04-23T18:34:01.225757+0000","last_active":"2024-04-23T18:34:02.337787+0000","last_peered":"2024-04-23T18:34:02.337787+0000","last_clean":"2024-04-23T18:34:02.337787+0000","last_became_active":"2024-04-23T18:34:01.225447+0000","last_became_peered":"2024-04-23T18:34:01.225447+0000","last_unstale":"2024-04-23T18:34:02.337787+0000","last_undegraded":"2024-04-23T18:34:02.337787+0000","last_fullsized":"2024-04-23T18:34:02.337787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:17.212135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.538554+0000","last_change":"2024-04-23T18:34:00.231749+0000","last_active":"2024-04-23T18:34:03.538554+0000","last_peered":"2024-04-23T18:34:03.538554+0000","last_clean":"2024-04-23T18:34:03.538554+0000","last_became_active":"2024-04-23T18:34:00.231207+0000","last_became_peered":"2024-04-23T18:34:00.231207+0000","last_unstale":"2024-04-23T18:34:03.538554+0000","last_undegraded":"2024-04-23T18:34:03.538554+0000","last_fullsized":"2024-04-23T18:34:03.538554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:23:05.723265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338077+0000","last_change":"2024-04-23T18:34:01.241116+0000","last_active":"2024-04-23T18:34:02.338077+0000","last_peered":"2024-04-23T18:34:02.338077+0000","last_clean":"2024-04-23T18:34:02.338077+0000","last_became_active":"2024-04-23T18:34:01.240785+0000","last_became_peered":"2024-04-23T18:34:01.240785+0000","last_unstale":"2024-04-23T18:34:02.338077+0000","last_undegraded":"2024-04-23T18:34:02.338077+0000","last_fullsized":"2024-04-23T18:34:02.338077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:48.323241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.548160+0000","last_change":"2024-04-23T18:34:00.228578+0000","last_active":"2024-04-23T18:34:04.548160+0000","last_peered":"2024-04-23T18:34:04.548160+0000","last_clean":"2024-04-23T18:34:04.548160+0000","last_became_active":"2024-04-23T18:34:00.228238+0000","last_became_peered":"2024-04-23T18:34:00.228238+0000","last_unstale":"2024-04-23T18:34:04.548160+0000","last_undegraded":"2024-04-23T18:34:04.548160+0000","last_fullsized":"2024-04-23T18:34:04.548160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:58:21.283593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337444+0000","last_change":"2024-04-23T18:34:01.239481+0000","last_active":"2024-04-23T18:34:02.337444+0000","last_peered":"2024-04-23T18:34:02.337444+0000","last_clean":"2024-04-23T18:34:02.337444+0000","last_became_active":"2024-04-23T18:34:01.239042+0000","last_became_peered":"2024-04-23T18:34:01.239042+0000","last_unstale":"2024-04-23T18:34:02.337444+0000","last_undegraded":"2024-04-23T18:34:02.337444+0000","last_fullsized":"2024-04-23T18:34:02.337444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:45.755462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:05.532325+0000","last_change":"2024-04-23T18:34:00.230015+0000","last_active":"2024-04-23T18:34:05.532325+0000","last_peered":"2024-04-23T18:34:05.532325+0000","last_clean":"2024-04-23T18:34:05.532325+0000","last_became_active":"2024-04-23T18:34:00.229672+0000","last_became_peered":"2024-04-23T18:34:00.229672+0000","last_unstale":"2024-04-23T18:34:05.532325+0000","last_undegraded":"2024-04-23T18:34:05.532325+0000","last_fullsized":"2024-04-23T18:34:05.532325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:19.478856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214206+0000","last_change":"2024-04-23T18:34:01.239943+0000","last_active":"2024-04-23T18:34:02.214206+0000","last_peered":"2024-04-23T18:34:02.214206+0000","last_clean":"2024-04-23T18:34:02.214206+0000","last_became_active":"2024-04-23T18:34:01.239588+0000","last_became_peered":"2024-04-23T18:34:01.239588+0000","last_unstale":"2024-04-23T18:34:02.214206+0000","last_undegraded":"2024-04-23T18:34:02.214206+0000","last_fullsized":"2024-04-23T18:34:02.214206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:51:39.344341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:05.520362+0000","last_change":"2024-04-23T18:34:00.227372+0000","last_active":"2024-04-23T18:34:05.520362+0000","last_peered":"2024-04-23T18:34:05.520362+0000","last_clean":"2024-04-23T18:34:05.520362+0000","last_became_active":"2024-04-23T18:34:00.226709+0000","last_became_peered":"2024-04-23T18:34:00.226709+0000","last_unstale":"2024-04-23T18:34:05.520362+0000","last_undegraded":"2024-04-23T18:34:05.520362+0000","last_fullsized":"2024-04-23T18:34:05.520362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:20.047108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337985+0000","last_change":"2024-04-23T18:34:01.235389+0000","last_active":"2024-04-23T18:34:02.337985+0000","last_peered":"2024-04-23T18:34:02.337985+0000","last_clean":"2024-04-23T18:34:02.337985+0000","last_became_active":"2024-04-23T18:34:01.235026+0000","last_became_peered":"2024-04-23T18:34:01.235026+0000","last_unstale":"2024-04-23T18:34:02.337985+0000","last_undegraded":"2024-04-23T18:34:02.337985+0000","last_fullsized":"2024-04-23T18:34:02.337985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:48.849525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337569+0000","last_change":"2024-04-23T18:34:00.236186+0000","last_active":"2024-04-23T18:34:02.337569+0000","last_peered":"2024-04-23T18:34:02.337569+0000","last_clean":"2024-04-23T18:34:02.337569+0000","last_became_active":"2024-04-23T18:34:00.235841+0000","last_became_peered":"2024-04-23T18:34:00.235841+0000","last_unstale":"2024-04-23T18:34:02.337569+0000","last_undegraded":"2024-04-23T18:34:02.337569+0000","last_fullsized":"2024-04-23T18:34:02.337569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:47.022715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.339192+0000","last_change":"2024-04-23T18:34:01.224992+0000","last_active":"2024-04-23T18:34:02.339192+0000","last_peered":"2024-04-23T18:34:02.339192+0000","last_clean":"2024-04-23T18:34:02.339192+0000","last_became_active":"2024-04-23T18:34:01.224508+0000","last_became_peered":"2024-04-23T18:34:01.224508+0000","last_unstale":"2024-04-23T18:34:02.339192+0000","last_undegraded":"2024-04-23T18:34:02.339192+0000","last_fullsized":"2024-04-23T18:34:02.339192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:24.183451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:06.545022+0000","last_change":"2024-04-23T18:34:00.234818+0000","last_active":"2024-04-23T18:34:06.545022+0000","last_peered":"2024-04-23T18:34:06.545022+0000","last_clean":"2024-04-23T18:34:06.545022+0000","last_became_active":"2024-04-23T18:34:00.234329+0000","last_became_peered":"2024-04-23T18:34:00.234329+0000","last_unstale":"2024-04-23T18:34:06.545022+0000","last_undegraded":"2024-04-23T18:34:06.545022+0000","last_fullsized":"2024-04-23T18:34:06.545022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:43:02.405694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338951+0000","last_change":"2024-04-23T18:34:01.227190+0000","last_active":"2024-04-23T18:34:02.338951+0000","last_peered":"2024-04-23T18:34:02.338951+0000","last_clean":"2024-04-23T18:34:02.338951+0000","last_became_active":"2024-04-23T18:34:01.226861+0000","last_became_peered":"2024-04-23T18:34:01.226861+0000","last_unstale":"2024-04-23T18:34:02.338951+0000","last_undegraded":"2024-04-23T18:34:02.338951+0000","last_fullsized":"2024-04-23T18:34:02.338951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:12.935643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214287+0000","last_change":"2024-04-23T18:34:01.234184+0000","last_active":"2024-04-23T18:34:02.214287+0000","last_peered":"2024-04-23T18:34:02.214287+0000","last_clean":"2024-04-23T18:34:02.214287+0000","last_became_active":"2024-04-23T18:34:01.233846+0000","last_became_peered":"2024-04-23T18:34:01.233846+0000","last_unstale":"2024-04-23T18:34:02.214287+0000","last_undegraded":"2024-04-23T18:34:02.214287+0000","last_fullsized":"2024-04-23T18:34:02.214287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:41.771495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.550304+0000","last_change":"2024-04-23T18:34:00.227862+0000","last_active":"2024-04-23T18:34:04.550304+0000","last_peered":"2024-04-23T18:34:04.550304+0000","last_clean":"2024-04-23T18:34:04.550304+0000","last_became_active":"2024-04-23T18:34:00.227515+0000","last_became_peered":"2024-04-23T18:34:00.227515+0000","last_unstale":"2024-04-23T18:34:04.550304+0000","last_undegraded":"2024-04-23T18:34:04.550304+0000","last_fullsized":"2024-04-23T18:34:04.550304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:18.595690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213803+0000","last_change":"2024-04-23T18:34:01.236338+0000","last_active":"2024-04-23T18:34:02.213803+0000","last_peered":"2024-04-23T18:34:02.213803+0000","last_clean":"2024-04-23T18:34:02.213803+0000","last_became_active":"2024-04-23T18:34:01.235795+0000","last_became_peered":"2024-04-23T18:34:01.235795+0000","last_unstale":"2024-04-23T18:34:02.213803+0000","last_undegraded":"2024-04-23T18:34:02.213803+0000","last_fullsized":"2024-04-23T18:34:02.213803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:45:01.928793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.538844+0000","last_change":"2024-04-23T18:34:00.221851+0000","last_active":"2024-04-23T18:34:03.538844+0000","last_peered":"2024-04-23T18:34:03.538844+0000","last_clean":"2024-04-23T18:34:03.538844+0000","last_became_active":"2024-04-23T18:34:00.221513+0000","last_became_peered":"2024-04-23T18:34:00.221513+0000","last_unstale":"2024-04-23T18:34:03.538844+0000","last_undegraded":"2024-04-23T18:34:03.538844+0000","last_fullsized":"2024-04-23T18:34:03.538844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:15:04.608759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338589+0000","last_change":"2024-04-23T18:32:48.218931+0000","last_active":"2024-04-23T18:34:02.338589+0000","last_peered":"2024-04-23T18:34:02.338589+0000","last_clean":"2024-04-23T18:34:02.338589+0000","last_became_active":"2024-04-23T18:32:48.218484+0000","last_became_peered":"2024-04-23T18:32:48.218484+0000","last_unstale":"2024-04-23T18:34:02.338589+0000","last_undegraded":"2024-04-23T18:34:02.338589+0000","last_fullsized":"2024-04-23T18:34:02.338589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_clean_scrub_stamp":"2024-04-23T18:29:21.029319+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:03:42.234165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337315+0000","last_change":"2024-04-23T18:34:01.220346+0000","last_active":"2024-04-23T18:34:02.337315+0000","last_peered":"2024-04-23T18:34:02.337315+0000","last_clean":"2024-04-23T18:34:02.337315+0000","last_became_active":"2024-04-23T18:34:01.219991+0000","last_became_peered":"2024-04-23T18:34:01.219991+0000","last_unstale":"2024-04-23T18:34:02.337315+0000","last_undegraded":"2024-04-23T18:34:02.337315+0000","last_fullsized":"2024-04-23T18:34:02.337315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:29:00.662700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.547985+0000","last_change":"2024-04-23T18:34:00.229625+0000","last_active":"2024-04-23T18:34:04.547985+0000","last_peered":"2024-04-23T18:34:04.547985+0000","last_clean":"2024-04-23T18:34:04.547985+0000","last_became_active":"2024-04-23T18:34:00.229204+0000","last_became_peered":"2024-04-23T18:34:00.229204+0000","last_unstale":"2024-04-23T18:34:04.547985+0000","last_undegraded":"2024-04-23T18:34:04.547985+0000","last_fullsized":"2024-04-23T18:34:04.547985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:42.370870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:05.532605+0000","last_change":"2024-04-23T18:34:00.236000+0000","last_active":"2024-04-23T18:34:05.532605+0000","last_peered":"2024-04-23T18:34:05.532605+0000","last_clean":"2024-04-23T18:34:05.532605+0000","last_became_active":"2024-04-23T18:34:00.235561+0000","last_became_peered":"2024-04-23T18:34:00.235561+0000","last_unstale":"2024-04-23T18:34:05.532605+0000","last_undegraded":"2024-04-23T18:34:05.532605+0000","last_fullsized":"2024-04-23T18:34:05.532605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:07.565489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338089+0000","last_change":"2024-04-23T18:34:01.227955+0000","last_active":"2024-04-23T18:34:02.338089+0000","last_peered":"2024-04-23T18:34:02.338089+0000","last_clean":"2024-04-23T18:34:02.338089+0000","last_became_active":"2024-04-23T18:34:01.225992+0000","last_became_peered":"2024-04-23T18:34:01.225992+0000","last_unstale":"2024-04-23T18:34:02.338089+0000","last_undegraded":"2024-04-23T18:34:02.338089+0000","last_fullsized":"2024-04-23T18:34:02.338089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:05:55.147008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337941+0000","last_change":"2024-04-23T18:34:00.230133+0000","last_active":"2024-04-23T18:34:02.337941+0000","last_peered":"2024-04-23T18:34:02.337941+0000","last_clean":"2024-04-23T18:34:02.337941+0000","last_became_active":"2024-04-23T18:34:00.229596+0000","last_became_peered":"2024-04-23T18:34:00.229596+0000","last_unstale":"2024-04-23T18:34:02.337941+0000","last_undegraded":"2024-04-23T18:34:02.337941+0000","last_fullsized":"2024-04-23T18:34:02.337941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:23:06.064313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214022+0000","last_change":"2024-04-23T18:34:01.229066+0000","last_active":"2024-04-23T18:34:02.214022+0000","last_peered":"2024-04-23T18:34:02.214022+0000","last_clean":"2024-04-23T18:34:02.214022+0000","last_became_active":"2024-04-23T18:34:01.228711+0000","last_became_peered":"2024-04-23T18:34:01.228711+0000","last_unstale":"2024-04-23T18:34:02.214022+0000","last_undegraded":"2024-04-23T18:34:02.214022+0000","last_fullsized":"2024-04-23T18:34:02.214022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:58:30.601270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.537920+0000","last_change":"2024-04-23T18:34:00.231227+0000","last_active":"2024-04-23T18:34:03.537920+0000","last_peered":"2024-04-23T18:34:03.537920+0000","last_clean":"2024-04-23T18:34:03.537920+0000","last_became_active":"2024-04-23T18:34:00.230843+0000","last_became_peered":"2024-04-23T18:34:00.230843+0000","last_unstale":"2024-04-23T18:34:03.537920+0000","last_undegraded":"2024-04-23T18:34:03.537920+0000","last_fullsized":"2024-04-23T18:34:03.537920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:55.416591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338277+0000","last_change":"2024-04-23T18:34:01.233164+0000","last_active":"2024-04-23T18:34:02.338277+0000","last_peered":"2024-04-23T18:34:02.338277+0000","last_clean":"2024-04-23T18:34:02.338277+0000","last_became_active":"2024-04-23T18:34:01.232827+0000","last_became_peered":"2024-04-23T18:34:01.232827+0000","last_unstale":"2024-04-23T18:34:02.338277+0000","last_undegraded":"2024-04-23T18:34:02.338277+0000","last_fullsized":"2024-04-23T18:34:02.338277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:09.635515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.538142+0000","last_change":"2024-04-23T18:34:00.225651+0000","last_active":"2024-04-23T18:34:03.538142+0000","last_peered":"2024-04-23T18:34:03.538142+0000","last_clean":"2024-04-23T18:34:03.538142+0000","last_became_active":"2024-04-23T18:34:00.225001+0000","last_became_peered":"2024-04-23T18:34:00.225001+0000","last_unstale":"2024-04-23T18:34:03.538142+0000","last_undegraded":"2024-04-23T18:34:03.538142+0000","last_fullsized":"2024-04-23T18:34:03.538142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:27:44.265388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.839790+0000","last_change":"2024-04-23T18:34:01.224131+0000","last_active":"2024-04-23T18:34:02.839790+0000","last_peered":"2024-04-23T18:34:02.839790+0000","last_clean":"2024-04-23T18:34:02.839790+0000","last_became_active":"2024-04-23T18:34:01.223742+0000","last_became_peered":"2024-04-23T18:34:01.223742+0000","last_unstale":"2024-04-23T18:34:02.839790+0000","last_undegraded":"2024-04-23T18:34:02.839790+0000","last_fullsized":"2024-04-23T18:34:02.839790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:29:26.430964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.538489+0000","last_change":"2024-04-23T18:34:00.235789+0000","last_active":"2024-04-23T18:34:03.538489+0000","last_peered":"2024-04-23T18:34:03.538489+0000","last_clean":"2024-04-23T18:34:03.538489+0000","last_became_active":"2024-04-23T18:34:00.235452+0000","last_became_peered":"2024-04-23T18:34:00.235452+0000","last_unstale":"2024-04-23T18:34:03.538489+0000","last_undegraded":"2024-04-23T18:34:03.538489+0000","last_fullsized":"2024-04-23T18:34:03.538489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:21.090121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337711+0000","last_change":"2024-04-23T18:34:01.226156+0000","last_active":"2024-04-23T18:34:02.337711+0000","last_peered":"2024-04-23T18:34:02.337711+0000","last_clean":"2024-04-23T18:34:02.337711+0000","last_became_active":"2024-04-23T18:34:01.225817+0000","last_became_peered":"2024-04-23T18:34:01.225817+0000","last_unstale":"2024-04-23T18:34:02.337711+0000","last_undegraded":"2024-04-23T18:34:02.337711+0000","last_fullsized":"2024-04-23T18:34:02.337711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:36.337139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337667+0000","last_change":"2024-04-23T18:34:00.235116+0000","last_active":"2024-04-23T18:34:02.337667+0000","last_peered":"2024-04-23T18:34:02.337667+0000","last_clean":"2024-04-23T18:34:02.337667+0000","last_became_active":"2024-04-23T18:34:00.234521+0000","last_became_peered":"2024-04-23T18:34:00.234521+0000","last_unstale":"2024-04-23T18:34:02.337667+0000","last_undegraded":"2024-04-23T18:34:02.337667+0000","last_fullsized":"2024-04-23T18:34:02.337667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:51.061164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213718+0000","last_change":"2024-04-23T18:34:01.226445+0000","last_active":"2024-04-23T18:34:02.213718+0000","last_peered":"2024-04-23T18:34:02.213718+0000","last_clean":"2024-04-23T18:34:02.213718+0000","last_became_active":"2024-04-23T18:34:01.225875+0000","last_became_peered":"2024-04-23T18:34:01.225875+0000","last_unstale":"2024-04-23T18:34:02.213718+0000","last_undegraded":"2024-04-23T18:34:02.213718+0000","last_fullsized":"2024-04-23T18:34:02.213718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:48:38.712502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.537975+0000","last_change":"2024-04-23T18:34:00.231918+0000","last_active":"2024-04-23T18:34:03.537975+0000","last_peered":"2024-04-23T18:34:03.537975+0000","last_clean":"2024-04-23T18:34:03.537975+0000","last_became_active":"2024-04-23T18:34:00.231550+0000","last_became_peered":"2024-04-23T18:34:00.231550+0000","last_unstale":"2024-04-23T18:34:03.537975+0000","last_undegraded":"2024-04-23T18:34:03.537975+0000","last_fullsized":"2024-04-23T18:34:03.537975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:52.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338142+0000","last_change":"2024-04-23T18:34:01.236360+0000","last_active":"2024-04-23T18:34:02.338142+0000","last_peered":"2024-04-23T18:34:02.338142+0000","last_clean":"2024-04-23T18:34:02.338142+0000","last_became_active":"2024-04-23T18:34:01.236056+0000","last_became_peered":"2024-04-23T18:34:01.236056+0000","last_unstale":"2024-04-23T18:34:02.338142+0000","last_undegraded":"2024-04-23T18:34:02.338142+0000","last_fullsized":"2024-04-23T18:34:02.338142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:54.535158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.549927+0000","last_change":"2024-04-23T18:34:00.233159+0000","last_active":"2024-04-23T18:34:04.549927+0000","last_peered":"2024-04-23T18:34:04.549927+0000","last_clean":"2024-04-23T18:34:04.549927+0000","last_became_active":"2024-04-23T18:34:00.232728+0000","last_became_peered":"2024-04-23T18:34:00.232728+0000","last_unstale":"2024-04-23T18:34:04.549927+0000","last_undegraded":"2024-04-23T18:34:04.549927+0000","last_fullsized":"2024-04-23T18:34:04.549927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:23.498057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338403+0000","last_change":"2024-04-23T18:34:01.226134+0000","last_active":"2024-04-23T18:34:02.338403+0000","last_peered":"2024-04-23T18:34:02.338403+0000","last_clean":"2024-04-23T18:34:02.338403+0000","last_became_active":"2024-04-23T18:34:01.225759+0000","last_became_peered":"2024-04-23T18:34:01.225759+0000","last_unstale":"2024-04-23T18:34:02.338403+0000","last_undegraded":"2024-04-23T18:34:02.338403+0000","last_fullsized":"2024-04-23T18:34:02.338403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:24.770595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214298+0000","last_change":"2024-04-23T18:34:00.230054+0000","last_active":"2024-04-23T18:34:02.214298+0000","last_peered":"2024-04-23T18:34:02.214298+0000","last_clean":"2024-04-23T18:34:02.214298+0000","last_became_active":"2024-04-23T18:34:00.229621+0000","last_became_peered":"2024-04-23T18:34:00.229621+0000","last_unstale":"2024-04-23T18:34:02.214298+0000","last_undegraded":"2024-04-23T18:34:02.214298+0000","last_fullsized":"2024-04-23T18:34:02.214298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:13:59.317282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338433+0000","last_change":"2024-04-23T18:34:01.240053+0000","last_active":"2024-04-23T18:34:02.338433+0000","last_peered":"2024-04-23T18:34:02.338433+0000","last_clean":"2024-04-23T18:34:02.338433+0000","last_became_active":"2024-04-23T18:34:01.239716+0000","last_became_peered":"2024-04-23T18:34:01.239716+0000","last_unstale":"2024-04-23T18:34:02.338433+0000","last_undegraded":"2024-04-23T18:34:02.338433+0000","last_fullsized":"2024-04-23T18:34:02.338433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:54.874374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337917+0000","last_change":"2024-04-23T18:34:00.227914+0000","last_active":"2024-04-23T18:34:02.337917+0000","last_peered":"2024-04-23T18:34:02.337917+0000","last_clean":"2024-04-23T18:34:02.337917+0000","last_became_active":"2024-04-23T18:34:00.227540+0000","last_became_peered":"2024-04-23T18:34:00.227540+0000","last_unstale":"2024-04-23T18:34:02.337917+0000","last_undegraded":"2024-04-23T18:34:02.337917+0000","last_fullsized":"2024-04-23T18:34:02.337917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:18.442397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338928+0000","last_change":"2024-04-23T18:34:01.227319+0000","last_active":"2024-04-23T18:34:02.338928+0000","last_peered":"2024-04-23T18:34:02.338928+0000","last_clean":"2024-04-23T18:34:02.338928+0000","last_became_active":"2024-04-23T18:34:01.226969+0000","last_became_peered":"2024-04-23T18:34:01.226969+0000","last_unstale":"2024-04-23T18:34:02.338928+0000","last_undegraded":"2024-04-23T18:34:02.338928+0000","last_fullsized":"2024-04-23T18:34:02.338928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:19.302388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337708+0000","last_change":"2024-04-23T18:34:01.234466+0000","last_active":"2024-04-23T18:34:02.337708+0000","last_peered":"2024-04-23T18:34:02.337708+0000","last_clean":"2024-04-23T18:34:02.337708+0000","last_became_active":"2024-04-23T18:34:01.234127+0000","last_became_peered":"2024-04-23T18:34:01.234127+0000","last_unstale":"2024-04-23T18:34:02.337708+0000","last_undegraded":"2024-04-23T18:34:02.337708+0000","last_fullsized":"2024-04-23T18:34:02.337708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:51:51.912320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338200+0000","last_change":"2024-04-23T18:34:01.229707+0000","last_active":"2024-04-23T18:34:02.338200+0000","last_peered":"2024-04-23T18:34:02.338200+0000","last_clean":"2024-04-23T18:34:02.338200+0000","last_became_active":"2024-04-23T18:34:01.229311+0000","last_became_peered":"2024-04-23T18:34:01.229311+0000","last_unstale":"2024-04-23T18:34:02.338200+0000","last_undegraded":"2024-04-23T18:34:02.338200+0000","last_fullsized":"2024-04-23T18:34:02.338200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:15:51.972436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338112+0000","last_change":"2024-04-23T18:34:01.227285+0000","last_active":"2024-04-23T18:34:02.338112+0000","last_peered":"2024-04-23T18:34:02.338112+0000","last_clean":"2024-04-23T18:34:02.338112+0000","last_became_active":"2024-04-23T18:34:01.226960+0000","last_became_peered":"2024-04-23T18:34:01.226960+0000","last_unstale":"2024-04-23T18:34:02.338112+0000","last_undegraded":"2024-04-23T18:34:02.338112+0000","last_fullsized":"2024-04-23T18:34:02.338112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:50.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337908+0000","last_change":"2024-04-23T18:34:01.230697+0000","last_active":"2024-04-23T18:34:02.337908+0000","last_peered":"2024-04-23T18:34:02.337908+0000","last_clean":"2024-04-23T18:34:02.337908+0000","last_became_active":"2024-04-23T18:34:01.230352+0000","last_became_peered":"2024-04-23T18:34:01.230352+0000","last_unstale":"2024-04-23T18:34:02.337908+0000","last_undegraded":"2024-04-23T18:34:02.337908+0000","last_fullsized":"2024-04-23T18:34:02.337908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:35:10.609773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338101+0000","last_change":"2024-04-23T18:34:01.226970+0000","last_active":"2024-04-23T18:34:02.338101+0000","last_peered":"2024-04-23T18:34:02.338101+0000","last_clean":"2024-04-23T18:34:02.338101+0000","last_became_active":"2024-04-23T18:34:01.224151+0000","last_became_peered":"2024-04-23T18:34:01.224151+0000","last_unstale":"2024-04-23T18:34:02.338101+0000","last_undegraded":"2024-04-23T18:34:02.338101+0000","last_fullsized":"2024-04-23T18:34:02.338101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:36:15.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338099+0000","last_change":"2024-04-23T18:34:01.234194+0000","last_active":"2024-04-23T18:34:02.338099+0000","last_peered":"2024-04-23T18:34:02.338099+0000","last_clean":"2024-04-23T18:34:02.338099+0000","last_became_active":"2024-04-23T18:34:01.232404+0000","last_became_peered":"2024-04-23T18:34:01.232404+0000","last_unstale":"2024-04-23T18:34:02.338099+0000","last_undegraded":"2024-04-23T18:34:02.338099+0000","last_fullsized":"2024-04-23T18:34:02.338099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:30.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338498+0000","last_change":"2024-04-23T18:34:01.219507+0000","last_active":"2024-04-23T18:34:02.338498+0000","last_peered":"2024-04-23T18:34:02.338498+0000","last_clean":"2024-04-23T18:34:02.338498+0000","last_became_active":"2024-04-23T18:34:01.218997+0000","last_became_peered":"2024-04-23T18:34:01.218997+0000","last_unstale":"2024-04-23T18:34:02.338498+0000","last_undegraded":"2024-04-23T18:34:02.338498+0000","last_fullsized":"2024-04-23T18:34:02.338498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:38.233652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213984+0000","last_change":"2024-04-23T18:34:01.231846+0000","last_active":"2024-04-23T18:34:02.213984+0000","last_peered":"2024-04-23T18:34:02.213984+0000","last_clean":"2024-04-23T18:34:02.213984+0000","last_became_active":"2024-04-23T18:34:01.231467+0000","last_became_peered":"2024-04-23T18:34:01.231467+0000","last_unstale":"2024-04-23T18:34:02.213984+0000","last_undegraded":"2024-04-23T18:34:02.213984+0000","last_fullsized":"2024-04-23T18:34:02.213984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:37:27.891180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214332+0000","last_change":"2024-04-23T18:34:01.237086+0000","last_active":"2024-04-23T18:34:02.214332+0000","last_peered":"2024-04-23T18:34:02.214332+0000","last_clean":"2024-04-23T18:34:02.214332+0000","last_became_active":"2024-04-23T18:34:01.236610+0000","last_became_peered":"2024-04-23T18:34:01.236610+0000","last_unstale":"2024-04-23T18:34:02.214332+0000","last_undegraded":"2024-04-23T18:34:02.214332+0000","last_fullsized":"2024-04-23T18:34:02.214332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:24.916553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213446+0000","last_change":"2024-04-23T18:34:01.231091+0000","last_active":"2024-04-23T18:34:02.213446+0000","last_peered":"2024-04-23T18:34:02.213446+0000","last_clean":"2024-04-23T18:34:02.213446+0000","last_became_active":"2024-04-23T18:34:01.230731+0000","last_became_peered":"2024-04-23T18:34:01.230731+0000","last_unstale":"2024-04-23T18:34:02.213446+0000","last_undegraded":"2024-04-23T18:34:02.213446+0000","last_fullsized":"2024-04-23T18:34:02.213446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:22.771515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338103+0000","last_change":"2024-04-23T18:34:01.231820+0000","last_active":"2024-04-23T18:34:02.338103+0000","last_peered":"2024-04-23T18:34:02.338103+0000","last_clean":"2024-04-23T18:34:02.338103+0000","last_became_active":"2024-04-23T18:34:01.230320+0000","last_became_peered":"2024-04-23T18:34:01.230320+0000","last_unstale":"2024-04-23T18:34:02.338103+0000","last_undegraded":"2024-04-23T18:34:02.338103+0000","last_fullsized":"2024-04-23T18:34:02.338103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:27:35.324323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338099+0000","last_change":"2024-04-23T18:34:01.228723+0000","last_active":"2024-04-23T18:34:02.338099+0000","last_peered":"2024-04-23T18:34:02.338099+0000","last_clean":"2024-04-23T18:34:02.338099+0000","last_became_active":"2024-04-23T18:34:01.227500+0000","last_became_peered":"2024-04-23T18:34:01.227500+0000","last_unstale":"2024-04-23T18:34:02.338099+0000","last_undegraded":"2024-04-23T18:34:02.338099+0000","last_fullsized":"2024-04-23T18:34:02.338099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:17.908021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337624+0000","last_change":"2024-04-23T18:34:01.227541+0000","last_active":"2024-04-23T18:34:02.337624+0000","last_peered":"2024-04-23T18:34:02.337624+0000","last_clean":"2024-04-23T18:34:02.337624+0000","last_became_active":"2024-04-23T18:34:01.227201+0000","last_became_peered":"2024-04-23T18:34:01.227201+0000","last_unstale":"2024-04-23T18:34:02.337624+0000","last_undegraded":"2024-04-23T18:34:02.337624+0000","last_fullsized":"2024-04-23T18:34:02.337624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:25:24.886850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282171,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6482,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":141,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":20137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123,"ondisk_log_size":123,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":445071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1037437,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":444663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":445379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":1037780,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":443879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":445491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":445945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":962560,"data_stored":446841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":443983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":444523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":1035723,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34247,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34247,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34247,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:34:08.908 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:34:09.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.049+0000 7f5b7519b640 1 -- 172.21.15.132:0/3502338068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b70101fe0 msgr2=0x7f5b70102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:09.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.049+0000 7f5b7519b640 1 --2- 172.21.15.132:0/3502338068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b70101fe0 0x7f5b70102460 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009900 tx=0x7f5b6002a410 comp rx=0 tx=0).stop 2024-04-23T18:34:09.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.050+0000 7f5b7519b640 1 -- 172.21.15.132:0/3502338068 shutdown_connections 2024-04-23T18:34:09.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.050+0000 7f5b7519b640 1 --2- 172.21.15.132:0/3502338068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b701029a0 0x7f5b701092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:09.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.050+0000 7f5b7519b640 1 --2- 172.21.15.132:0/3502338068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b70101fe0 0x7f5b70102460 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:09.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.050+0000 7f5b7519b640 1 --2- 172.21.15.132:0/3502338068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b70100e90 0x7f5b701012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:09.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.050+0000 7f5b7519b640 1 -- 172.21.15.132:0/3502338068 >> 172.21.15.132:0/3502338068 conn(0x7f5b700fc440 msgr2=0x7f5b700fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:09.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.050+0000 7f5b7519b640 1 -- 172.21.15.132:0/3502338068 shutdown_connections 2024-04-23T18:34:09.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.050+0000 7f5b7519b640 1 -- 172.21.15.132:0/3502338068 wait complete. 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.050+0000 7f5b7519b640 1 Processor -- start 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.050+0000 7f5b7519b640 1 -- start start 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b7519b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b70100e90 0x7f5b7006b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b7519b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b70101fe0 0x7f5b7019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b7519b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b701029a0 0x7f5b7019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b7519b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010c9d0 con 0x7f5b70101fe0 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b7519b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010c850 con 0x7f5b70100e90 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b7519b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010cb50 con 0x7f5b701029a0 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b6e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b70101fe0 0x7f5b7019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b6e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b70101fe0 0x7f5b7019cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36178/0 (socket says 172.21.15.132:36178) 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b6e59c640 1 -- 172.21.15.132:0/466830813 learned_addr learned my addr 172.21.15.132:0/466830813 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b6ed9d640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b70100e90 0x7f5b7006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:09.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b6e59c640 1 -- 172.21.15.132:0/466830813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b701029a0 msgr2=0x7f5b7019f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:09.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b6e59c640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b701029a0 0x7f5b7019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:09.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b6e59c640 1 -- 172.21.15.132:0/466830813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b70100e90 msgr2=0x7f5b7006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:09.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b6e59c640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b70100e90 0x7f5b7006b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:09.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b6e59c640 1 -- 172.21.15.132:0/466830813 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7019fe40 con 0x7f5b70101fe0 2024-04-23T18:34:09.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b6e59c640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b70101fe0 0x7f5b7019cf10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5b60006010 tx=0x7f5b600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:09.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b57fff640 1 -- 172.21.15.132:0/466830813 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6003f030 con 0x7f5b70101fe0 2024-04-23T18:34:09.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b57fff640 1 -- 172.21.15.132:0/466830813 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b60003cb0 con 0x7f5b70101fe0 2024-04-23T18:34:09.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b7519b640 1 -- 172.21.15.132:0/466830813 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b701a0070 con 0x7f5b70101fe0 2024-04-23T18:34:09.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b57fff640 1 -- 172.21.15.132:0/466830813 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60035070 con 0x7f5b70101fe0 2024-04-23T18:34:09.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.051+0000 7f5b7519b640 1 -- 172.21.15.132:0/466830813 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b701a0560 con 0x7f5b70101fe0 2024-04-23T18:34:09.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.052+0000 7f5b7519b640 1 -- 172.21.15.132:0/466830813 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b70108290 con 0x7f5b70101fe0 2024-04-23T18:34:09.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.053+0000 7f5b57fff640 1 -- 172.21.15.132:0/466830813 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5b60042030 con 0x7f5b70101fe0 2024-04-23T18:34:09.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.054+0000 7f5b57fff640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5b580a3c40 0x7f5b580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:09.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.054+0000 7f5b6ed9d640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5b580a3c40 0x7f5b580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:09.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.054+0000 7f5b57fff640 1 -- 172.21.15.132:0/466830813 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5b600f0ad0 con 0x7f5b70101fe0 2024-04-23T18:34:09.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.054+0000 7f5b6ed9d640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5b580a3c40 0x7f5b580a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b640058c0 tx=0x7f5b6400a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:09.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.058+0000 7f5b57fff640 1 -- 172.21.15.132:0/466830813 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b600bd500 con 0x7f5b70101fe0 2024-04-23T18:34:09.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.236+0000 7f5b7519b640 1 -- 172.21.15.132:0/466830813 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b70062390 con 0x7f5b580a3c40 2024-04-23T18:34:09.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.244+0000 7f5b57fff640 1 -- 172.21.15.132:0/466830813 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258947 (secure 0 0 0) 0x7f5b70062390 con 0x7f5b580a3c40 2024-04-23T18:34:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:34:09.240 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-04-23T18:34:09.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.247+0000 7f5b7519b640 1 -- 172.21.15.132:0/466830813 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5b580a3c40 msgr2=0x7f5b580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:09.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.247+0000 7f5b7519b640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5b580a3c40 0x7f5b580a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b640058c0 tx=0x7f5b6400a4c0 comp rx=0 tx=0).stop 2024-04-23T18:34:09.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.247+0000 7f5b7519b640 1 -- 172.21.15.132:0/466830813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b70101fe0 msgr2=0x7f5b7019cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:09.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.247+0000 7f5b7519b640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b70101fe0 0x7f5b7019cf10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5b60006010 tx=0x7f5b600042f0 comp rx=0 tx=0).stop 2024-04-23T18:34:09.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.247+0000 7f5b7519b640 1 -- 172.21.15.132:0/466830813 shutdown_connections 2024-04-23T18:34:09.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.247+0000 7f5b7519b640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5b580a3c40 0x7f5b580a6100 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:09.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.247+0000 7f5b7519b640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b701029a0 0x7f5b7019f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:09.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.247+0000 7f5b7519b640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b70101fe0 0x7f5b7019cf10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:09.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.247+0000 7f5b7519b640 1 --2- 172.21.15.132:0/466830813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b70100e90 0x7f5b7006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:09.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.247+0000 7f5b7519b640 1 -- 172.21.15.132:0/466830813 >> 172.21.15.132:0/466830813 conn(0x7f5b700fc440 msgr2=0x7f5b70105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:09.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.247+0000 7f5b7519b640 1 -- 172.21.15.132:0/466830813 shutdown_connections 2024-04-23T18:34:09.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:09.247+0000 7f5b7519b640 1 -- 172.21.15.132:0/466830813 wait complete. 2024-04-23T18:34:09.251 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-04-23T18:34:08.125447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596850,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":144,"num_read_kb":127,"num_write":322,"num_write_kb":2979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":260,"ondisk_log_size":260,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352116,"kb_used_data":11256,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589708,"statfs":{"total":1151940427776,"available":1151579860992,"internally_reserved":0,"allocated":11526144,"data_stored":7116715,"data_compressed":107658,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6482,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":141,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.915297"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338134+0000","last_change":"2024-04-23T18:34:01.231204+0000","last_active":"2024-04-23T18:34:02.338134+0000","last_peered":"2024-04-23T18:34:02.338134+0000","last_clean":"2024-04-23T18:34:02.338134+0000","last_became_active":"2024-04-23T18:34:01.230836+0000","last_became_peered":"2024-04-23T18:34:01.230836+0000","last_unstale":"2024-04-23T18:34:02.338134+0000","last_undegraded":"2024-04-23T18:34:02.338134+0000","last_fullsized":"2024-04-23T18:34:02.338134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:57.744807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338435+0000","last_change":"2024-04-23T18:34:01.225833+0000","last_active":"2024-04-23T18:34:02.338435+0000","last_peered":"2024-04-23T18:34:02.338435+0000","last_clean":"2024-04-23T18:34:02.338435+0000","last_became_active":"2024-04-23T18:34:01.225098+0000","last_became_peered":"2024-04-23T18:34:01.225098+0000","last_unstale":"2024-04-23T18:34:02.338435+0000","last_undegraded":"2024-04-23T18:34:02.338435+0000","last_fullsized":"2024-04-23T18:34:02.338435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:20.720137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338461+0000","last_change":"2024-04-23T18:34:01.235269+0000","last_active":"2024-04-23T18:34:02.338461+0000","last_peered":"2024-04-23T18:34:02.338461+0000","last_clean":"2024-04-23T18:34:02.338461+0000","last_became_active":"2024-04-23T18:34:01.234921+0000","last_became_peered":"2024-04-23T18:34:01.234921+0000","last_unstale":"2024-04-23T18:34:02.338461+0000","last_undegraded":"2024-04-23T18:34:02.338461+0000","last_fullsized":"2024-04-23T18:34:02.338461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:06:20.645693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214493+0000","last_change":"2024-04-23T18:34:01.233316+0000","last_active":"2024-04-23T18:34:02.214493+0000","last_peered":"2024-04-23T18:34:02.214493+0000","last_clean":"2024-04-23T18:34:02.214493+0000","last_became_active":"2024-04-23T18:34:01.231636+0000","last_became_peered":"2024-04-23T18:34:01.231636+0000","last_unstale":"2024-04-23T18:34:02.214493+0000","last_undegraded":"2024-04-23T18:34:02.214493+0000","last_fullsized":"2024-04-23T18:34:02.214493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:15.837129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.839838+0000","last_change":"2024-04-23T18:34:01.228073+0000","last_active":"2024-04-23T18:34:02.839838+0000","last_peered":"2024-04-23T18:34:02.839838+0000","last_clean":"2024-04-23T18:34:02.839838+0000","last_became_active":"2024-04-23T18:34:01.227701+0000","last_became_peered":"2024-04-23T18:34:01.227701+0000","last_unstale":"2024-04-23T18:34:02.839838+0000","last_undegraded":"2024-04-23T18:34:02.839838+0000","last_fullsized":"2024-04-23T18:34:02.839838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:17:26.556498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338952+0000","last_change":"2024-04-23T18:34:01.228033+0000","last_active":"2024-04-23T18:34:02.338952+0000","last_peered":"2024-04-23T18:34:02.338952+0000","last_clean":"2024-04-23T18:34:02.338952+0000","last_became_active":"2024-04-23T18:34:01.227665+0000","last_became_peered":"2024-04-23T18:34:01.227665+0000","last_unstale":"2024-04-23T18:34:02.338952+0000","last_undegraded":"2024-04-23T18:34:02.338952+0000","last_fullsized":"2024-04-23T18:34:02.338952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:53:07.663983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337775+0000","last_change":"2024-04-23T18:34:01.238484+0000","last_active":"2024-04-23T18:34:02.337775+0000","last_peered":"2024-04-23T18:34:02.337775+0000","last_clean":"2024-04-23T18:34:02.337775+0000","last_became_active":"2024-04-23T18:34:01.238091+0000","last_became_peered":"2024-04-23T18:34:01.238091+0000","last_unstale":"2024-04-23T18:34:02.337775+0000","last_undegraded":"2024-04-23T18:34:02.337775+0000","last_fullsized":"2024-04-23T18:34:02.337775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:52.472116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213438+0000","last_change":"2024-04-23T18:34:01.233934+0000","last_active":"2024-04-23T18:34:02.213438+0000","last_peered":"2024-04-23T18:34:02.213438+0000","last_clean":"2024-04-23T18:34:02.213438+0000","last_became_active":"2024-04-23T18:34:01.233597+0000","last_became_peered":"2024-04-23T18:34:01.233597+0000","last_unstale":"2024-04-23T18:34:02.213438+0000","last_undegraded":"2024-04-23T18:34:02.213438+0000","last_fullsized":"2024-04-23T18:34:02.213438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:33:51.911420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338587+0000","last_change":"2024-04-23T18:34:01.234094+0000","last_active":"2024-04-23T18:34:02.338587+0000","last_peered":"2024-04-23T18:34:02.338587+0000","last_clean":"2024-04-23T18:34:02.338587+0000","last_became_active":"2024-04-23T18:34:01.232499+0000","last_became_peered":"2024-04-23T18:34:01.232499+0000","last_unstale":"2024-04-23T18:34:02.338587+0000","last_undegraded":"2024-04-23T18:34:02.338587+0000","last_fullsized":"2024-04-23T18:34:02.338587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:33.004245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.339353+0000","last_change":"2024-04-23T18:34:01.229175+0000","last_active":"2024-04-23T18:34:02.339353+0000","last_peered":"2024-04-23T18:34:02.339353+0000","last_clean":"2024-04-23T18:34:02.339353+0000","last_became_active":"2024-04-23T18:34:01.228837+0000","last_became_peered":"2024-04-23T18:34:01.228837+0000","last_unstale":"2024-04-23T18:34:02.339353+0000","last_undegraded":"2024-04-23T18:34:02.339353+0000","last_fullsized":"2024-04-23T18:34:02.339353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:57.625110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338125+0000","last_change":"2024-04-23T18:34:01.235557+0000","last_active":"2024-04-23T18:34:02.338125+0000","last_peered":"2024-04-23T18:34:02.338125+0000","last_clean":"2024-04-23T18:34:02.338125+0000","last_became_active":"2024-04-23T18:34:01.235184+0000","last_became_peered":"2024-04-23T18:34:01.235184+0000","last_unstale":"2024-04-23T18:34:02.338125+0000","last_undegraded":"2024-04-23T18:34:02.338125+0000","last_fullsized":"2024-04-23T18:34:02.338125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:37:27.138041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214421+0000","last_change":"2024-04-23T18:34:01.234070+0000","last_active":"2024-04-23T18:34:02.214421+0000","last_peered":"2024-04-23T18:34:02.214421+0000","last_clean":"2024-04-23T18:34:02.214421+0000","last_became_active":"2024-04-23T18:34:01.233727+0000","last_became_peered":"2024-04-23T18:34:01.233727+0000","last_unstale":"2024-04-23T18:34:02.214421+0000","last_undegraded":"2024-04-23T18:34:02.214421+0000","last_fullsized":"2024-04-23T18:34:02.214421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:59:16.489819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338473+0000","last_change":"2024-04-23T18:34:01.228256+0000","last_active":"2024-04-23T18:34:02.338473+0000","last_peered":"2024-04-23T18:34:02.338473+0000","last_clean":"2024-04-23T18:34:02.338473+0000","last_became_active":"2024-04-23T18:34:01.224400+0000","last_became_peered":"2024-04-23T18:34:01.224400+0000","last_unstale":"2024-04-23T18:34:02.338473+0000","last_undegraded":"2024-04-23T18:34:02.338473+0000","last_fullsized":"2024-04-23T18:34:02.338473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:37:28.401695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337550+0000","last_change":"2024-04-23T18:34:01.227250+0000","last_active":"2024-04-23T18:34:02.337550+0000","last_peered":"2024-04-23T18:34:02.337550+0000","last_clean":"2024-04-23T18:34:02.337550+0000","last_became_active":"2024-04-23T18:34:01.226937+0000","last_became_peered":"2024-04-23T18:34:01.226937+0000","last_unstale":"2024-04-23T18:34:02.337550+0000","last_undegraded":"2024-04-23T18:34:02.337550+0000","last_fullsized":"2024-04-23T18:34:02.337550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:26:20.801691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337432+0000","last_change":"2024-04-23T18:34:01.232351+0000","last_active":"2024-04-23T18:34:02.337432+0000","last_peered":"2024-04-23T18:34:02.337432+0000","last_clean":"2024-04-23T18:34:02.337432+0000","last_became_active":"2024-04-23T18:34:01.231987+0000","last_became_peered":"2024-04-23T18:34:01.231987+0000","last_unstale":"2024-04-23T18:34:02.337432+0000","last_undegraded":"2024-04-23T18:34:02.337432+0000","last_fullsized":"2024-04-23T18:34:02.337432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:50.886496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337521+0000","last_change":"2024-04-23T18:34:01.232196+0000","last_active":"2024-04-23T18:34:02.337521+0000","last_peered":"2024-04-23T18:34:02.337521+0000","last_clean":"2024-04-23T18:34:02.337521+0000","last_became_active":"2024-04-23T18:34:01.231746+0000","last_became_peered":"2024-04-23T18:34:01.231746+0000","last_unstale":"2024-04-23T18:34:02.337521+0000","last_undegraded":"2024-04-23T18:34:02.337521+0000","last_fullsized":"2024-04-23T18:34:02.337521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:58.888549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338302+0000","last_change":"2024-04-23T18:34:01.231509+0000","last_active":"2024-04-23T18:34:02.338302+0000","last_peered":"2024-04-23T18:34:02.338302+0000","last_clean":"2024-04-23T18:34:02.338302+0000","last_became_active":"2024-04-23T18:34:01.231017+0000","last_became_peered":"2024-04-23T18:34:01.231017+0000","last_unstale":"2024-04-23T18:34:02.338302+0000","last_undegraded":"2024-04-23T18:34:02.338302+0000","last_fullsized":"2024-04-23T18:34:02.338302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:04:51.929462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338089+0000","last_change":"2024-04-23T18:34:01.223285+0000","last_active":"2024-04-23T18:34:02.338089+0000","last_peered":"2024-04-23T18:34:02.338089+0000","last_clean":"2024-04-23T18:34:02.338089+0000","last_became_active":"2024-04-23T18:34:01.222962+0000","last_became_peered":"2024-04-23T18:34:01.222962+0000","last_unstale":"2024-04-23T18:34:02.338089+0000","last_undegraded":"2024-04-23T18:34:02.338089+0000","last_fullsized":"2024-04-23T18:34:02.338089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:18.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338026+0000","last_change":"2024-04-23T18:34:01.225922+0000","last_active":"2024-04-23T18:34:02.338026+0000","last_peered":"2024-04-23T18:34:02.338026+0000","last_clean":"2024-04-23T18:34:02.338026+0000","last_became_active":"2024-04-23T18:34:01.225235+0000","last_became_peered":"2024-04-23T18:34:01.225235+0000","last_unstale":"2024-04-23T18:34:02.338026+0000","last_undegraded":"2024-04-23T18:34:02.338026+0000","last_fullsized":"2024-04-23T18:34:02.338026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:33.480782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337427+0000","last_change":"2024-04-23T18:34:00.224606+0000","last_active":"2024-04-23T18:34:02.337427+0000","last_peered":"2024-04-23T18:34:02.337427+0000","last_clean":"2024-04-23T18:34:02.337427+0000","last_became_active":"2024-04-23T18:34:00.224291+0000","last_became_peered":"2024-04-23T18:34:00.224291+0000","last_unstale":"2024-04-23T18:34:02.337427+0000","last_undegraded":"2024-04-23T18:34:02.337427+0000","last_fullsized":"2024-04-23T18:34:02.337427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:40.734704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213549+0000","last_change":"2024-04-23T18:34:01.235023+0000","last_active":"2024-04-23T18:34:02.213549+0000","last_peered":"2024-04-23T18:34:02.213549+0000","last_clean":"2024-04-23T18:34:02.213549+0000","last_became_active":"2024-04-23T18:34:01.234672+0000","last_became_peered":"2024-04-23T18:34:01.234672+0000","last_unstale":"2024-04-23T18:34:02.213549+0000","last_undegraded":"2024-04-23T18:34:02.213549+0000","last_fullsized":"2024-04-23T18:34:02.213549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:33.734343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213943+0000","last_change":"2024-04-23T18:34:00.232552+0000","last_active":"2024-04-23T18:34:02.213943+0000","last_peered":"2024-04-23T18:34:02.213943+0000","last_clean":"2024-04-23T18:34:02.213943+0000","last_became_active":"2024-04-23T18:34:00.232226+0000","last_became_peered":"2024-04-23T18:34:00.232226+0000","last_unstale":"2024-04-23T18:34:02.213943+0000","last_undegraded":"2024-04-23T18:34:02.213943+0000","last_fullsized":"2024-04-23T18:34:02.213943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.563757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338458+0000","last_change":"2024-04-23T18:34:01.236013+0000","last_active":"2024-04-23T18:34:02.338458+0000","last_peered":"2024-04-23T18:34:02.338458+0000","last_clean":"2024-04-23T18:34:02.338458+0000","last_became_active":"2024-04-23T18:34:01.235590+0000","last_became_peered":"2024-04-23T18:34:01.235590+0000","last_unstale":"2024-04-23T18:34:02.338458+0000","last_undegraded":"2024-04-23T18:34:02.338458+0000","last_fullsized":"2024-04-23T18:34:02.338458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:45.741648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:07.539158+0000","last_change":"2024-04-23T18:34:00.232336+0000","last_active":"2024-04-23T18:34:07.539158+0000","last_peered":"2024-04-23T18:34:07.539158+0000","last_clean":"2024-04-23T18:34:07.539158+0000","last_became_active":"2024-04-23T18:34:00.231839+0000","last_became_peered":"2024-04-23T18:34:00.231839+0000","last_unstale":"2024-04-23T18:34:07.539158+0000","last_undegraded":"2024-04-23T18:34:07.539158+0000","last_fullsized":"2024-04-23T18:34:07.539158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:38.029434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338067+0000","last_change":"2024-04-23T18:34:01.230435+0000","last_active":"2024-04-23T18:34:02.338067+0000","last_peered":"2024-04-23T18:34:02.338067+0000","last_clean":"2024-04-23T18:34:02.338067+0000","last_became_active":"2024-04-23T18:34:01.230095+0000","last_became_peered":"2024-04-23T18:34:01.230095+0000","last_unstale":"2024-04-23T18:34:02.338067+0000","last_undegraded":"2024-04-23T18:34:02.338067+0000","last_fullsized":"2024-04-23T18:34:02.338067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:31.653254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:06.545234+0000","last_change":"2024-04-23T18:34:00.227853+0000","last_active":"2024-04-23T18:34:06.545234+0000","last_peered":"2024-04-23T18:34:06.545234+0000","last_clean":"2024-04-23T18:34:06.545234+0000","last_became_active":"2024-04-23T18:34:00.226596+0000","last_became_peered":"2024-04-23T18:34:00.226596+0000","last_unstale":"2024-04-23T18:34:06.545234+0000","last_undegraded":"2024-04-23T18:34:06.545234+0000","last_fullsized":"2024-04-23T18:34:06.545234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:16:10.077969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214447+0000","last_change":"2024-04-23T18:34:01.235955+0000","last_active":"2024-04-23T18:34:02.214447+0000","last_peered":"2024-04-23T18:34:02.214447+0000","last_clean":"2024-04-23T18:34:02.214447+0000","last_became_active":"2024-04-23T18:34:01.235609+0000","last_became_peered":"2024-04-23T18:34:01.235609+0000","last_unstale":"2024-04-23T18:34:02.214447+0000","last_undegraded":"2024-04-23T18:34:02.214447+0000","last_fullsized":"2024-04-23T18:34:02.214447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:07:39.480213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.541425+0000","last_change":"2024-04-23T18:34:00.226319+0000","last_active":"2024-04-23T18:34:03.541425+0000","last_peered":"2024-04-23T18:34:03.541425+0000","last_clean":"2024-04-23T18:34:03.541425+0000","last_became_active":"2024-04-23T18:34:00.225796+0000","last_became_peered":"2024-04-23T18:34:00.225796+0000","last_unstale":"2024-04-23T18:34:03.541425+0000","last_undegraded":"2024-04-23T18:34:03.541425+0000","last_fullsized":"2024-04-23T18:34:03.541425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:11:18.309181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.839858+0000","last_change":"2024-04-23T18:34:01.227336+0000","last_active":"2024-04-23T18:34:02.839858+0000","last_peered":"2024-04-23T18:34:02.839858+0000","last_clean":"2024-04-23T18:34:02.839858+0000","last_became_active":"2024-04-23T18:34:01.226987+0000","last_became_peered":"2024-04-23T18:34:01.226987+0000","last_unstale":"2024-04-23T18:34:02.839858+0000","last_undegraded":"2024-04-23T18:34:02.839858+0000","last_fullsized":"2024-04-23T18:34:02.839858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:22:30.058673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.548594+0000","last_change":"2024-04-23T18:34:00.222748+0000","last_active":"2024-04-23T18:34:04.548594+0000","last_peered":"2024-04-23T18:34:04.548594+0000","last_clean":"2024-04-23T18:34:04.548594+0000","last_became_active":"2024-04-23T18:34:00.222412+0000","last_became_peered":"2024-04-23T18:34:00.222412+0000","last_unstale":"2024-04-23T18:34:04.548594+0000","last_undegraded":"2024-04-23T18:34:04.548594+0000","last_fullsized":"2024-04-23T18:34:04.548594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:48:24.807152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214270+0000","last_change":"2024-04-23T18:34:01.234354+0000","last_active":"2024-04-23T18:34:02.214270+0000","last_peered":"2024-04-23T18:34:02.214270+0000","last_clean":"2024-04-23T18:34:02.214270+0000","last_became_active":"2024-04-23T18:34:01.233815+0000","last_became_peered":"2024-04-23T18:34:01.233815+0000","last_unstale":"2024-04-23T18:34:02.214270+0000","last_undegraded":"2024-04-23T18:34:02.214270+0000","last_fullsized":"2024-04-23T18:34:02.214270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:58.793701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.550170+0000","last_change":"2024-04-23T18:34:00.227595+0000","last_active":"2024-04-23T18:34:04.550170+0000","last_peered":"2024-04-23T18:34:04.550170+0000","last_clean":"2024-04-23T18:34:04.550170+0000","last_became_active":"2024-04-23T18:34:00.226830+0000","last_became_peered":"2024-04-23T18:34:00.226830+0000","last_unstale":"2024-04-23T18:34:04.550170+0000","last_undegraded":"2024-04-23T18:34:04.550170+0000","last_fullsized":"2024-04-23T18:34:04.550170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:20.661984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213674+0000","last_change":"2024-04-23T18:34:01.225053+0000","last_active":"2024-04-23T18:34:02.213674+0000","last_peered":"2024-04-23T18:34:02.213674+0000","last_clean":"2024-04-23T18:34:02.213674+0000","last_became_active":"2024-04-23T18:34:01.224723+0000","last_became_peered":"2024-04-23T18:34:01.224723+0000","last_unstale":"2024-04-23T18:34:02.213674+0000","last_undegraded":"2024-04-23T18:34:02.213674+0000","last_fullsized":"2024-04-23T18:34:02.213674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:01:51.486691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.537458+0000","last_change":"2024-04-23T18:34:00.224783+0000","last_active":"2024-04-23T18:34:03.537458+0000","last_peered":"2024-04-23T18:34:03.537458+0000","last_clean":"2024-04-23T18:34:03.537458+0000","last_became_active":"2024-04-23T18:34:00.224436+0000","last_became_peered":"2024-04-23T18:34:00.224436+0000","last_unstale":"2024-04-23T18:34:03.537458+0000","last_undegraded":"2024-04-23T18:34:03.537458+0000","last_fullsized":"2024-04-23T18:34:03.537458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:33.829188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337541+0000","last_change":"2024-04-23T18:34:01.224283+0000","last_active":"2024-04-23T18:34:02.337541+0000","last_peered":"2024-04-23T18:34:02.337541+0000","last_clean":"2024-04-23T18:34:02.337541+0000","last_became_active":"2024-04-23T18:34:01.223949+0000","last_became_peered":"2024-04-23T18:34:01.223949+0000","last_unstale":"2024-04-23T18:34:02.337541+0000","last_undegraded":"2024-04-23T18:34:02.337541+0000","last_fullsized":"2024-04-23T18:34:02.337541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:50.560687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.539367+0000","last_change":"2024-04-23T18:34:00.218453+0000","last_active":"2024-04-23T18:34:03.539367+0000","last_peered":"2024-04-23T18:34:03.539367+0000","last_clean":"2024-04-23T18:34:03.539367+0000","last_became_active":"2024-04-23T18:34:00.218036+0000","last_became_peered":"2024-04-23T18:34:00.218036+0000","last_unstale":"2024-04-23T18:34:03.539367+0000","last_undegraded":"2024-04-23T18:34:03.539367+0000","last_fullsized":"2024-04-23T18:34:03.539367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:49.485185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338427+0000","last_change":"2024-04-23T18:34:01.231143+0000","last_active":"2024-04-23T18:34:02.338427+0000","last_peered":"2024-04-23T18:34:02.338427+0000","last_clean":"2024-04-23T18:34:02.338427+0000","last_became_active":"2024-04-23T18:34:01.230812+0000","last_became_peered":"2024-04-23T18:34:01.230812+0000","last_unstale":"2024-04-23T18:34:02.338427+0000","last_undegraded":"2024-04-23T18:34:02.338427+0000","last_fullsized":"2024-04-23T18:34:02.338427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:28.704807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:06.544251+0000","last_change":"2024-04-23T18:34:00.235711+0000","last_active":"2024-04-23T18:34:06.544251+0000","last_peered":"2024-04-23T18:34:06.544251+0000","last_clean":"2024-04-23T18:34:06.544251+0000","last_became_active":"2024-04-23T18:34:00.235333+0000","last_became_peered":"2024-04-23T18:34:00.235333+0000","last_unstale":"2024-04-23T18:34:06.544251+0000","last_undegraded":"2024-04-23T18:34:06.544251+0000","last_fullsized":"2024-04-23T18:34:06.544251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:51.320744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338458+0000","last_change":"2024-04-23T18:34:01.238291+0000","last_active":"2024-04-23T18:34:02.338458+0000","last_peered":"2024-04-23T18:34:02.338458+0000","last_clean":"2024-04-23T18:34:02.338458+0000","last_became_active":"2024-04-23T18:34:01.237972+0000","last_became_peered":"2024-04-23T18:34:01.237972+0000","last_unstale":"2024-04-23T18:34:02.338458+0000","last_undegraded":"2024-04-23T18:34:02.338458+0000","last_fullsized":"2024-04-23T18:34:02.338458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:36:17.108170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.538485+0000","last_change":"2024-04-23T18:34:00.227249+0000","last_active":"2024-04-23T18:34:03.538485+0000","last_peered":"2024-04-23T18:34:03.538485+0000","last_clean":"2024-04-23T18:34:03.538485+0000","last_became_active":"2024-04-23T18:34:00.226854+0000","last_became_peered":"2024-04-23T18:34:00.226854+0000","last_unstale":"2024-04-23T18:34:03.538485+0000","last_undegraded":"2024-04-23T18:34:03.538485+0000","last_fullsized":"2024-04-23T18:34:03.538485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:49.619906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338126+0000","last_change":"2024-04-23T18:34:01.225518+0000","last_active":"2024-04-23T18:34:02.338126+0000","last_peered":"2024-04-23T18:34:02.338126+0000","last_clean":"2024-04-23T18:34:02.338126+0000","last_became_active":"2024-04-23T18:34:01.224907+0000","last_became_peered":"2024-04-23T18:34:01.224907+0000","last_unstale":"2024-04-23T18:34:02.338126+0000","last_undegraded":"2024-04-23T18:34:02.338126+0000","last_fullsized":"2024-04-23T18:34:02.338126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:19.876629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.538881+0000","last_change":"2024-04-23T18:34:00.224770+0000","last_active":"2024-04-23T18:34:03.538881+0000","last_peered":"2024-04-23T18:34:03.538881+0000","last_clean":"2024-04-23T18:34:03.538881+0000","last_became_active":"2024-04-23T18:34:00.224395+0000","last_became_peered":"2024-04-23T18:34:00.224395+0000","last_unstale":"2024-04-23T18:34:03.538881+0000","last_undegraded":"2024-04-23T18:34:03.538881+0000","last_fullsized":"2024-04-23T18:34:03.538881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:14:09.172479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.840122+0000","last_change":"2024-04-23T18:34:01.227771+0000","last_active":"2024-04-23T18:34:02.840122+0000","last_peered":"2024-04-23T18:34:02.840122+0000","last_clean":"2024-04-23T18:34:02.840122+0000","last_became_active":"2024-04-23T18:34:01.225063+0000","last_became_peered":"2024-04-23T18:34:01.225063+0000","last_unstale":"2024-04-23T18:34:02.840122+0000","last_undegraded":"2024-04-23T18:34:02.840122+0000","last_fullsized":"2024-04-23T18:34:02.840122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:53.921587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.547953+0000","last_change":"2024-04-23T18:34:00.230455+0000","last_active":"2024-04-23T18:34:04.547953+0000","last_peered":"2024-04-23T18:34:04.547953+0000","last_clean":"2024-04-23T18:34:04.547953+0000","last_became_active":"2024-04-23T18:34:00.229934+0000","last_became_peered":"2024-04-23T18:34:00.229934+0000","last_unstale":"2024-04-23T18:34:04.547953+0000","last_undegraded":"2024-04-23T18:34:04.547953+0000","last_fullsized":"2024-04-23T18:34:04.547953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:26:34.787606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337787+0000","last_change":"2024-04-23T18:34:01.225757+0000","last_active":"2024-04-23T18:34:02.337787+0000","last_peered":"2024-04-23T18:34:02.337787+0000","last_clean":"2024-04-23T18:34:02.337787+0000","last_became_active":"2024-04-23T18:34:01.225447+0000","last_became_peered":"2024-04-23T18:34:01.225447+0000","last_unstale":"2024-04-23T18:34:02.337787+0000","last_undegraded":"2024-04-23T18:34:02.337787+0000","last_fullsized":"2024-04-23T18:34:02.337787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:17.212135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.538554+0000","last_change":"2024-04-23T18:34:00.231749+0000","last_active":"2024-04-23T18:34:03.538554+0000","last_peered":"2024-04-23T18:34:03.538554+0000","last_clean":"2024-04-23T18:34:03.538554+0000","last_became_active":"2024-04-23T18:34:00.231207+0000","last_became_peered":"2024-04-23T18:34:00.231207+0000","last_unstale":"2024-04-23T18:34:03.538554+0000","last_undegraded":"2024-04-23T18:34:03.538554+0000","last_fullsized":"2024-04-23T18:34:03.538554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:23:05.723265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338077+0000","last_change":"2024-04-23T18:34:01.241116+0000","last_active":"2024-04-23T18:34:02.338077+0000","last_peered":"2024-04-23T18:34:02.338077+0000","last_clean":"2024-04-23T18:34:02.338077+0000","last_became_active":"2024-04-23T18:34:01.240785+0000","last_became_peered":"2024-04-23T18:34:01.240785+0000","last_unstale":"2024-04-23T18:34:02.338077+0000","last_undegraded":"2024-04-23T18:34:02.338077+0000","last_fullsized":"2024-04-23T18:34:02.338077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:48.323241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.548160+0000","last_change":"2024-04-23T18:34:00.228578+0000","last_active":"2024-04-23T18:34:04.548160+0000","last_peered":"2024-04-23T18:34:04.548160+0000","last_clean":"2024-04-23T18:34:04.548160+0000","last_became_active":"2024-04-23T18:34:00.228238+0000","last_became_peered":"2024-04-23T18:34:00.228238+0000","last_unstale":"2024-04-23T18:34:04.548160+0000","last_undegraded":"2024-04-23T18:34:04.548160+0000","last_fullsized":"2024-04-23T18:34:04.548160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:58:21.283593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337444+0000","last_change":"2024-04-23T18:34:01.239481+0000","last_active":"2024-04-23T18:34:02.337444+0000","last_peered":"2024-04-23T18:34:02.337444+0000","last_clean":"2024-04-23T18:34:02.337444+0000","last_became_active":"2024-04-23T18:34:01.239042+0000","last_became_peered":"2024-04-23T18:34:01.239042+0000","last_unstale":"2024-04-23T18:34:02.337444+0000","last_undegraded":"2024-04-23T18:34:02.337444+0000","last_fullsized":"2024-04-23T18:34:02.337444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:45.755462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:05.532325+0000","last_change":"2024-04-23T18:34:00.230015+0000","last_active":"2024-04-23T18:34:05.532325+0000","last_peered":"2024-04-23T18:34:05.532325+0000","last_clean":"2024-04-23T18:34:05.532325+0000","last_became_active":"2024-04-23T18:34:00.229672+0000","last_became_peered":"2024-04-23T18:34:00.229672+0000","last_unstale":"2024-04-23T18:34:05.532325+0000","last_undegraded":"2024-04-23T18:34:05.532325+0000","last_fullsized":"2024-04-23T18:34:05.532325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:19.478856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214206+0000","last_change":"2024-04-23T18:34:01.239943+0000","last_active":"2024-04-23T18:34:02.214206+0000","last_peered":"2024-04-23T18:34:02.214206+0000","last_clean":"2024-04-23T18:34:02.214206+0000","last_became_active":"2024-04-23T18:34:01.239588+0000","last_became_peered":"2024-04-23T18:34:01.239588+0000","last_unstale":"2024-04-23T18:34:02.214206+0000","last_undegraded":"2024-04-23T18:34:02.214206+0000","last_fullsized":"2024-04-23T18:34:02.214206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:51:39.344341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:05.520362+0000","last_change":"2024-04-23T18:34:00.227372+0000","last_active":"2024-04-23T18:34:05.520362+0000","last_peered":"2024-04-23T18:34:05.520362+0000","last_clean":"2024-04-23T18:34:05.520362+0000","last_became_active":"2024-04-23T18:34:00.226709+0000","last_became_peered":"2024-04-23T18:34:00.226709+0000","last_unstale":"2024-04-23T18:34:05.520362+0000","last_undegraded":"2024-04-23T18:34:05.520362+0000","last_fullsized":"2024-04-23T18:34:05.520362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:20.047108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337985+0000","last_change":"2024-04-23T18:34:01.235389+0000","last_active":"2024-04-23T18:34:02.337985+0000","last_peered":"2024-04-23T18:34:02.337985+0000","last_clean":"2024-04-23T18:34:02.337985+0000","last_became_active":"2024-04-23T18:34:01.235026+0000","last_became_peered":"2024-04-23T18:34:01.235026+0000","last_unstale":"2024-04-23T18:34:02.337985+0000","last_undegraded":"2024-04-23T18:34:02.337985+0000","last_fullsized":"2024-04-23T18:34:02.337985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:48.849525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337569+0000","last_change":"2024-04-23T18:34:00.236186+0000","last_active":"2024-04-23T18:34:02.337569+0000","last_peered":"2024-04-23T18:34:02.337569+0000","last_clean":"2024-04-23T18:34:02.337569+0000","last_became_active":"2024-04-23T18:34:00.235841+0000","last_became_peered":"2024-04-23T18:34:00.235841+0000","last_unstale":"2024-04-23T18:34:02.337569+0000","last_undegraded":"2024-04-23T18:34:02.337569+0000","last_fullsized":"2024-04-23T18:34:02.337569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:47.022715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.339192+0000","last_change":"2024-04-23T18:34:01.224992+0000","last_active":"2024-04-23T18:34:02.339192+0000","last_peered":"2024-04-23T18:34:02.339192+0000","last_clean":"2024-04-23T18:34:02.339192+0000","last_became_active":"2024-04-23T18:34:01.224508+0000","last_became_peered":"2024-04-23T18:34:01.224508+0000","last_unstale":"2024-04-23T18:34:02.339192+0000","last_undegraded":"2024-04-23T18:34:02.339192+0000","last_fullsized":"2024-04-23T18:34:02.339192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:24.183451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:06.545022+0000","last_change":"2024-04-23T18:34:00.234818+0000","last_active":"2024-04-23T18:34:06.545022+0000","last_peered":"2024-04-23T18:34:06.545022+0000","last_clean":"2024-04-23T18:34:06.545022+0000","last_became_active":"2024-04-23T18:34:00.234329+0000","last_became_peered":"2024-04-23T18:34:00.234329+0000","last_unstale":"2024-04-23T18:34:06.545022+0000","last_undegraded":"2024-04-23T18:34:06.545022+0000","last_fullsized":"2024-04-23T18:34:06.545022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:43:02.405694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338951+0000","last_change":"2024-04-23T18:34:01.227190+0000","last_active":"2024-04-23T18:34:02.338951+0000","last_peered":"2024-04-23T18:34:02.338951+0000","last_clean":"2024-04-23T18:34:02.338951+0000","last_became_active":"2024-04-23T18:34:01.226861+0000","last_became_peered":"2024-04-23T18:34:01.226861+0000","last_unstale":"2024-04-23T18:34:02.338951+0000","last_undegraded":"2024-04-23T18:34:02.338951+0000","last_fullsized":"2024-04-23T18:34:02.338951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:12.935643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214287+0000","last_change":"2024-04-23T18:34:01.234184+0000","last_active":"2024-04-23T18:34:02.214287+0000","last_peered":"2024-04-23T18:34:02.214287+0000","last_clean":"2024-04-23T18:34:02.214287+0000","last_became_active":"2024-04-23T18:34:01.233846+0000","last_became_peered":"2024-04-23T18:34:01.233846+0000","last_unstale":"2024-04-23T18:34:02.214287+0000","last_undegraded":"2024-04-23T18:34:02.214287+0000","last_fullsized":"2024-04-23T18:34:02.214287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:41.771495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.550304+0000","last_change":"2024-04-23T18:34:00.227862+0000","last_active":"2024-04-23T18:34:04.550304+0000","last_peered":"2024-04-23T18:34:04.550304+0000","last_clean":"2024-04-23T18:34:04.550304+0000","last_became_active":"2024-04-23T18:34:00.227515+0000","last_became_peered":"2024-04-23T18:34:00.227515+0000","last_unstale":"2024-04-23T18:34:04.550304+0000","last_undegraded":"2024-04-23T18:34:04.550304+0000","last_fullsized":"2024-04-23T18:34:04.550304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:18.595690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213803+0000","last_change":"2024-04-23T18:34:01.236338+0000","last_active":"2024-04-23T18:34:02.213803+0000","last_peered":"2024-04-23T18:34:02.213803+0000","last_clean":"2024-04-23T18:34:02.213803+0000","last_became_active":"2024-04-23T18:34:01.235795+0000","last_became_peered":"2024-04-23T18:34:01.235795+0000","last_unstale":"2024-04-23T18:34:02.213803+0000","last_undegraded":"2024-04-23T18:34:02.213803+0000","last_fullsized":"2024-04-23T18:34:02.213803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:45:01.928793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.538844+0000","last_change":"2024-04-23T18:34:00.221851+0000","last_active":"2024-04-23T18:34:03.538844+0000","last_peered":"2024-04-23T18:34:03.538844+0000","last_clean":"2024-04-23T18:34:03.538844+0000","last_became_active":"2024-04-23T18:34:00.221513+0000","last_became_peered":"2024-04-23T18:34:00.221513+0000","last_unstale":"2024-04-23T18:34:03.538844+0000","last_undegraded":"2024-04-23T18:34:03.538844+0000","last_fullsized":"2024-04-23T18:34:03.538844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:15:04.608759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338589+0000","last_change":"2024-04-23T18:32:48.218931+0000","last_active":"2024-04-23T18:34:02.338589+0000","last_peered":"2024-04-23T18:34:02.338589+0000","last_clean":"2024-04-23T18:34:02.338589+0000","last_became_active":"2024-04-23T18:32:48.218484+0000","last_became_peered":"2024-04-23T18:32:48.218484+0000","last_unstale":"2024-04-23T18:34:02.338589+0000","last_undegraded":"2024-04-23T18:34:02.338589+0000","last_fullsized":"2024-04-23T18:34:02.338589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_clean_scrub_stamp":"2024-04-23T18:29:21.029319+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:03:42.234165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337315+0000","last_change":"2024-04-23T18:34:01.220346+0000","last_active":"2024-04-23T18:34:02.337315+0000","last_peered":"2024-04-23T18:34:02.337315+0000","last_clean":"2024-04-23T18:34:02.337315+0000","last_became_active":"2024-04-23T18:34:01.219991+0000","last_became_peered":"2024-04-23T18:34:01.219991+0000","last_unstale":"2024-04-23T18:34:02.337315+0000","last_undegraded":"2024-04-23T18:34:02.337315+0000","last_fullsized":"2024-04-23T18:34:02.337315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:29:00.662700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.547985+0000","last_change":"2024-04-23T18:34:00.229625+0000","last_active":"2024-04-23T18:34:04.547985+0000","last_peered":"2024-04-23T18:34:04.547985+0000","last_clean":"2024-04-23T18:34:04.547985+0000","last_became_active":"2024-04-23T18:34:00.229204+0000","last_became_peered":"2024-04-23T18:34:00.229204+0000","last_unstale":"2024-04-23T18:34:04.547985+0000","last_undegraded":"2024-04-23T18:34:04.547985+0000","last_fullsized":"2024-04-23T18:34:04.547985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:42.370870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:05.532605+0000","last_change":"2024-04-23T18:34:00.236000+0000","last_active":"2024-04-23T18:34:05.532605+0000","last_peered":"2024-04-23T18:34:05.532605+0000","last_clean":"2024-04-23T18:34:05.532605+0000","last_became_active":"2024-04-23T18:34:00.235561+0000","last_became_peered":"2024-04-23T18:34:00.235561+0000","last_unstale":"2024-04-23T18:34:05.532605+0000","last_undegraded":"2024-04-23T18:34:05.532605+0000","last_fullsized":"2024-04-23T18:34:05.532605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:07.565489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338089+0000","last_change":"2024-04-23T18:34:01.227955+0000","last_active":"2024-04-23T18:34:02.338089+0000","last_peered":"2024-04-23T18:34:02.338089+0000","last_clean":"2024-04-23T18:34:02.338089+0000","last_became_active":"2024-04-23T18:34:01.225992+0000","last_became_peered":"2024-04-23T18:34:01.225992+0000","last_unstale":"2024-04-23T18:34:02.338089+0000","last_undegraded":"2024-04-23T18:34:02.338089+0000","last_fullsized":"2024-04-23T18:34:02.338089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:05:55.147008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337941+0000","last_change":"2024-04-23T18:34:00.230133+0000","last_active":"2024-04-23T18:34:02.337941+0000","last_peered":"2024-04-23T18:34:02.337941+0000","last_clean":"2024-04-23T18:34:02.337941+0000","last_became_active":"2024-04-23T18:34:00.229596+0000","last_became_peered":"2024-04-23T18:34:00.229596+0000","last_unstale":"2024-04-23T18:34:02.337941+0000","last_undegraded":"2024-04-23T18:34:02.337941+0000","last_fullsized":"2024-04-23T18:34:02.337941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:23:06.064313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214022+0000","last_change":"2024-04-23T18:34:01.229066+0000","last_active":"2024-04-23T18:34:02.214022+0000","last_peered":"2024-04-23T18:34:02.214022+0000","last_clean":"2024-04-23T18:34:02.214022+0000","last_became_active":"2024-04-23T18:34:01.228711+0000","last_became_peered":"2024-04-23T18:34:01.228711+0000","last_unstale":"2024-04-23T18:34:02.214022+0000","last_undegraded":"2024-04-23T18:34:02.214022+0000","last_fullsized":"2024-04-23T18:34:02.214022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:58:30.601270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.537920+0000","last_change":"2024-04-23T18:34:00.231227+0000","last_active":"2024-04-23T18:34:03.537920+0000","last_peered":"2024-04-23T18:34:03.537920+0000","last_clean":"2024-04-23T18:34:03.537920+0000","last_became_active":"2024-04-23T18:34:00.230843+0000","last_became_peered":"2024-04-23T18:34:00.230843+0000","last_unstale":"2024-04-23T18:34:03.537920+0000","last_undegraded":"2024-04-23T18:34:03.537920+0000","last_fullsized":"2024-04-23T18:34:03.537920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:55.416591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338277+0000","last_change":"2024-04-23T18:34:01.233164+0000","last_active":"2024-04-23T18:34:02.338277+0000","last_peered":"2024-04-23T18:34:02.338277+0000","last_clean":"2024-04-23T18:34:02.338277+0000","last_became_active":"2024-04-23T18:34:01.232827+0000","last_became_peered":"2024-04-23T18:34:01.232827+0000","last_unstale":"2024-04-23T18:34:02.338277+0000","last_undegraded":"2024-04-23T18:34:02.338277+0000","last_fullsized":"2024-04-23T18:34:02.338277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:09.635515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.538142+0000","last_change":"2024-04-23T18:34:00.225651+0000","last_active":"2024-04-23T18:34:03.538142+0000","last_peered":"2024-04-23T18:34:03.538142+0000","last_clean":"2024-04-23T18:34:03.538142+0000","last_became_active":"2024-04-23T18:34:00.225001+0000","last_became_peered":"2024-04-23T18:34:00.225001+0000","last_unstale":"2024-04-23T18:34:03.538142+0000","last_undegraded":"2024-04-23T18:34:03.538142+0000","last_fullsized":"2024-04-23T18:34:03.538142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:27:44.265388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.839790+0000","last_change":"2024-04-23T18:34:01.224131+0000","last_active":"2024-04-23T18:34:02.839790+0000","last_peered":"2024-04-23T18:34:02.839790+0000","last_clean":"2024-04-23T18:34:02.839790+0000","last_became_active":"2024-04-23T18:34:01.223742+0000","last_became_peered":"2024-04-23T18:34:01.223742+0000","last_unstale":"2024-04-23T18:34:02.839790+0000","last_undegraded":"2024-04-23T18:34:02.839790+0000","last_fullsized":"2024-04-23T18:34:02.839790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:29:26.430964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.538489+0000","last_change":"2024-04-23T18:34:00.235789+0000","last_active":"2024-04-23T18:34:03.538489+0000","last_peered":"2024-04-23T18:34:03.538489+0000","last_clean":"2024-04-23T18:34:03.538489+0000","last_became_active":"2024-04-23T18:34:00.235452+0000","last_became_peered":"2024-04-23T18:34:00.235452+0000","last_unstale":"2024-04-23T18:34:03.538489+0000","last_undegraded":"2024-04-23T18:34:03.538489+0000","last_fullsized":"2024-04-23T18:34:03.538489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:21.090121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337711+0000","last_change":"2024-04-23T18:34:01.226156+0000","last_active":"2024-04-23T18:34:02.337711+0000","last_peered":"2024-04-23T18:34:02.337711+0000","last_clean":"2024-04-23T18:34:02.337711+0000","last_became_active":"2024-04-23T18:34:01.225817+0000","last_became_peered":"2024-04-23T18:34:01.225817+0000","last_unstale":"2024-04-23T18:34:02.337711+0000","last_undegraded":"2024-04-23T18:34:02.337711+0000","last_fullsized":"2024-04-23T18:34:02.337711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:36.337139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337667+0000","last_change":"2024-04-23T18:34:00.235116+0000","last_active":"2024-04-23T18:34:02.337667+0000","last_peered":"2024-04-23T18:34:02.337667+0000","last_clean":"2024-04-23T18:34:02.337667+0000","last_became_active":"2024-04-23T18:34:00.234521+0000","last_became_peered":"2024-04-23T18:34:00.234521+0000","last_unstale":"2024-04-23T18:34:02.337667+0000","last_undegraded":"2024-04-23T18:34:02.337667+0000","last_fullsized":"2024-04-23T18:34:02.337667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:51.061164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213718+0000","last_change":"2024-04-23T18:34:01.226445+0000","last_active":"2024-04-23T18:34:02.213718+0000","last_peered":"2024-04-23T18:34:02.213718+0000","last_clean":"2024-04-23T18:34:02.213718+0000","last_became_active":"2024-04-23T18:34:01.225875+0000","last_became_peered":"2024-04-23T18:34:01.225875+0000","last_unstale":"2024-04-23T18:34:02.213718+0000","last_undegraded":"2024-04-23T18:34:02.213718+0000","last_fullsized":"2024-04-23T18:34:02.213718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:48:38.712502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:03.537975+0000","last_change":"2024-04-23T18:34:00.231918+0000","last_active":"2024-04-23T18:34:03.537975+0000","last_peered":"2024-04-23T18:34:03.537975+0000","last_clean":"2024-04-23T18:34:03.537975+0000","last_became_active":"2024-04-23T18:34:00.231550+0000","last_became_peered":"2024-04-23T18:34:00.231550+0000","last_unstale":"2024-04-23T18:34:03.537975+0000","last_undegraded":"2024-04-23T18:34:03.537975+0000","last_fullsized":"2024-04-23T18:34:03.537975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:52.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338142+0000","last_change":"2024-04-23T18:34:01.236360+0000","last_active":"2024-04-23T18:34:02.338142+0000","last_peered":"2024-04-23T18:34:02.338142+0000","last_clean":"2024-04-23T18:34:02.338142+0000","last_became_active":"2024-04-23T18:34:01.236056+0000","last_became_peered":"2024-04-23T18:34:01.236056+0000","last_unstale":"2024-04-23T18:34:02.338142+0000","last_undegraded":"2024-04-23T18:34:02.338142+0000","last_fullsized":"2024-04-23T18:34:02.338142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:54.535158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:04.549927+0000","last_change":"2024-04-23T18:34:00.233159+0000","last_active":"2024-04-23T18:34:04.549927+0000","last_peered":"2024-04-23T18:34:04.549927+0000","last_clean":"2024-04-23T18:34:04.549927+0000","last_became_active":"2024-04-23T18:34:00.232728+0000","last_became_peered":"2024-04-23T18:34:00.232728+0000","last_unstale":"2024-04-23T18:34:04.549927+0000","last_undegraded":"2024-04-23T18:34:04.549927+0000","last_fullsized":"2024-04-23T18:34:04.549927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:23.498057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338403+0000","last_change":"2024-04-23T18:34:01.226134+0000","last_active":"2024-04-23T18:34:02.338403+0000","last_peered":"2024-04-23T18:34:02.338403+0000","last_clean":"2024-04-23T18:34:02.338403+0000","last_became_active":"2024-04-23T18:34:01.225759+0000","last_became_peered":"2024-04-23T18:34:01.225759+0000","last_unstale":"2024-04-23T18:34:02.338403+0000","last_undegraded":"2024-04-23T18:34:02.338403+0000","last_fullsized":"2024-04-23T18:34:02.338403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:24.770595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214298+0000","last_change":"2024-04-23T18:34:00.230054+0000","last_active":"2024-04-23T18:34:02.214298+0000","last_peered":"2024-04-23T18:34:02.214298+0000","last_clean":"2024-04-23T18:34:02.214298+0000","last_became_active":"2024-04-23T18:34:00.229621+0000","last_became_peered":"2024-04-23T18:34:00.229621+0000","last_unstale":"2024-04-23T18:34:02.214298+0000","last_undegraded":"2024-04-23T18:34:02.214298+0000","last_fullsized":"2024-04-23T18:34:02.214298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:13:59.317282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338433+0000","last_change":"2024-04-23T18:34:01.240053+0000","last_active":"2024-04-23T18:34:02.338433+0000","last_peered":"2024-04-23T18:34:02.338433+0000","last_clean":"2024-04-23T18:34:02.338433+0000","last_became_active":"2024-04-23T18:34:01.239716+0000","last_became_peered":"2024-04-23T18:34:01.239716+0000","last_unstale":"2024-04-23T18:34:02.338433+0000","last_undegraded":"2024-04-23T18:34:02.338433+0000","last_fullsized":"2024-04-23T18:34:02.338433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:54.874374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337917+0000","last_change":"2024-04-23T18:34:00.227914+0000","last_active":"2024-04-23T18:34:02.337917+0000","last_peered":"2024-04-23T18:34:02.337917+0000","last_clean":"2024-04-23T18:34:02.337917+0000","last_became_active":"2024-04-23T18:34:00.227540+0000","last_became_peered":"2024-04-23T18:34:00.227540+0000","last_unstale":"2024-04-23T18:34:02.337917+0000","last_undegraded":"2024-04-23T18:34:02.337917+0000","last_fullsized":"2024-04-23T18:34:02.337917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:18.442397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338928+0000","last_change":"2024-04-23T18:34:01.227319+0000","last_active":"2024-04-23T18:34:02.338928+0000","last_peered":"2024-04-23T18:34:02.338928+0000","last_clean":"2024-04-23T18:34:02.338928+0000","last_became_active":"2024-04-23T18:34:01.226969+0000","last_became_peered":"2024-04-23T18:34:01.226969+0000","last_unstale":"2024-04-23T18:34:02.338928+0000","last_undegraded":"2024-04-23T18:34:02.338928+0000","last_fullsized":"2024-04-23T18:34:02.338928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:19.302388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337708+0000","last_change":"2024-04-23T18:34:01.234466+0000","last_active":"2024-04-23T18:34:02.337708+0000","last_peered":"2024-04-23T18:34:02.337708+0000","last_clean":"2024-04-23T18:34:02.337708+0000","last_became_active":"2024-04-23T18:34:01.234127+0000","last_became_peered":"2024-04-23T18:34:01.234127+0000","last_unstale":"2024-04-23T18:34:02.337708+0000","last_undegraded":"2024-04-23T18:34:02.337708+0000","last_fullsized":"2024-04-23T18:34:02.337708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:51:51.912320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338200+0000","last_change":"2024-04-23T18:34:01.229707+0000","last_active":"2024-04-23T18:34:02.338200+0000","last_peered":"2024-04-23T18:34:02.338200+0000","last_clean":"2024-04-23T18:34:02.338200+0000","last_became_active":"2024-04-23T18:34:01.229311+0000","last_became_peered":"2024-04-23T18:34:01.229311+0000","last_unstale":"2024-04-23T18:34:02.338200+0000","last_undegraded":"2024-04-23T18:34:02.338200+0000","last_fullsized":"2024-04-23T18:34:02.338200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:15:51.972436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338112+0000","last_change":"2024-04-23T18:34:01.227285+0000","last_active":"2024-04-23T18:34:02.338112+0000","last_peered":"2024-04-23T18:34:02.338112+0000","last_clean":"2024-04-23T18:34:02.338112+0000","last_became_active":"2024-04-23T18:34:01.226960+0000","last_became_peered":"2024-04-23T18:34:01.226960+0000","last_unstale":"2024-04-23T18:34:02.338112+0000","last_undegraded":"2024-04-23T18:34:02.338112+0000","last_fullsized":"2024-04-23T18:34:02.338112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:50.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337908+0000","last_change":"2024-04-23T18:34:01.230697+0000","last_active":"2024-04-23T18:34:02.337908+0000","last_peered":"2024-04-23T18:34:02.337908+0000","last_clean":"2024-04-23T18:34:02.337908+0000","last_became_active":"2024-04-23T18:34:01.230352+0000","last_became_peered":"2024-04-23T18:34:01.230352+0000","last_unstale":"2024-04-23T18:34:02.337908+0000","last_undegraded":"2024-04-23T18:34:02.337908+0000","last_fullsized":"2024-04-23T18:34:02.337908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:35:10.609773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338101+0000","last_change":"2024-04-23T18:34:01.226970+0000","last_active":"2024-04-23T18:34:02.338101+0000","last_peered":"2024-04-23T18:34:02.338101+0000","last_clean":"2024-04-23T18:34:02.338101+0000","last_became_active":"2024-04-23T18:34:01.224151+0000","last_became_peered":"2024-04-23T18:34:01.224151+0000","last_unstale":"2024-04-23T18:34:02.338101+0000","last_undegraded":"2024-04-23T18:34:02.338101+0000","last_fullsized":"2024-04-23T18:34:02.338101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:36:15.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338099+0000","last_change":"2024-04-23T18:34:01.234194+0000","last_active":"2024-04-23T18:34:02.338099+0000","last_peered":"2024-04-23T18:34:02.338099+0000","last_clean":"2024-04-23T18:34:02.338099+0000","last_became_active":"2024-04-23T18:34:01.232404+0000","last_became_peered":"2024-04-23T18:34:01.232404+0000","last_unstale":"2024-04-23T18:34:02.338099+0000","last_undegraded":"2024-04-23T18:34:02.338099+0000","last_fullsized":"2024-04-23T18:34:02.338099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:30.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338498+0000","last_change":"2024-04-23T18:34:01.219507+0000","last_active":"2024-04-23T18:34:02.338498+0000","last_peered":"2024-04-23T18:34:02.338498+0000","last_clean":"2024-04-23T18:34:02.338498+0000","last_became_active":"2024-04-23T18:34:01.218997+0000","last_became_peered":"2024-04-23T18:34:01.218997+0000","last_unstale":"2024-04-23T18:34:02.338498+0000","last_undegraded":"2024-04-23T18:34:02.338498+0000","last_fullsized":"2024-04-23T18:34:02.338498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:38.233652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213984+0000","last_change":"2024-04-23T18:34:01.231846+0000","last_active":"2024-04-23T18:34:02.213984+0000","last_peered":"2024-04-23T18:34:02.213984+0000","last_clean":"2024-04-23T18:34:02.213984+0000","last_became_active":"2024-04-23T18:34:01.231467+0000","last_became_peered":"2024-04-23T18:34:01.231467+0000","last_unstale":"2024-04-23T18:34:02.213984+0000","last_undegraded":"2024-04-23T18:34:02.213984+0000","last_fullsized":"2024-04-23T18:34:02.213984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:37:27.891180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.214332+0000","last_change":"2024-04-23T18:34:01.237086+0000","last_active":"2024-04-23T18:34:02.214332+0000","last_peered":"2024-04-23T18:34:02.214332+0000","last_clean":"2024-04-23T18:34:02.214332+0000","last_became_active":"2024-04-23T18:34:01.236610+0000","last_became_peered":"2024-04-23T18:34:01.236610+0000","last_unstale":"2024-04-23T18:34:02.214332+0000","last_undegraded":"2024-04-23T18:34:02.214332+0000","last_fullsized":"2024-04-23T18:34:02.214332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:24.916553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.213446+0000","last_change":"2024-04-23T18:34:01.231091+0000","last_active":"2024-04-23T18:34:02.213446+0000","last_peered":"2024-04-23T18:34:02.213446+0000","last_clean":"2024-04-23T18:34:02.213446+0000","last_became_active":"2024-04-23T18:34:01.230731+0000","last_became_peered":"2024-04-23T18:34:01.230731+0000","last_unstale":"2024-04-23T18:34:02.213446+0000","last_undegraded":"2024-04-23T18:34:02.213446+0000","last_fullsized":"2024-04-23T18:34:02.213446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:22.771515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338103+0000","last_change":"2024-04-23T18:34:01.231820+0000","last_active":"2024-04-23T18:34:02.338103+0000","last_peered":"2024-04-23T18:34:02.338103+0000","last_clean":"2024-04-23T18:34:02.338103+0000","last_became_active":"2024-04-23T18:34:01.230320+0000","last_became_peered":"2024-04-23T18:34:01.230320+0000","last_unstale":"2024-04-23T18:34:02.338103+0000","last_undegraded":"2024-04-23T18:34:02.338103+0000","last_fullsized":"2024-04-23T18:34:02.338103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:27:35.324323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.338099+0000","last_change":"2024-04-23T18:34:01.228723+0000","last_active":"2024-04-23T18:34:02.338099+0000","last_peered":"2024-04-23T18:34:02.338099+0000","last_clean":"2024-04-23T18:34:02.338099+0000","last_became_active":"2024-04-23T18:34:01.227500+0000","last_became_peered":"2024-04-23T18:34:01.227500+0000","last_unstale":"2024-04-23T18:34:02.338099+0000","last_undegraded":"2024-04-23T18:34:02.338099+0000","last_fullsized":"2024-04-23T18:34:02.338099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:17.908021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T18:34:02.337624+0000","last_change":"2024-04-23T18:34:01.227541+0000","last_active":"2024-04-23T18:34:02.337624+0000","last_peered":"2024-04-23T18:34:02.337624+0000","last_clean":"2024-04-23T18:34:02.337624+0000","last_became_active":"2024-04-23T18:34:01.227201+0000","last_became_peered":"2024-04-23T18:34:01.227201+0000","last_unstale":"2024-04-23T18:34:02.337624+0000","last_undegraded":"2024-04-23T18:34:02.337624+0000","last_fullsized":"2024-04-23T18:34:02.337624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:25:24.886850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282171,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6482,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":141,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":20137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123,"ondisk_log_size":123,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":445071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1037437,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":444663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":445379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":1037780,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":443879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":445491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":445945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":962560,"data_stored":446841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":443983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":444523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":1035723,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34247,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34247,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34247,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:34:09.255 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T18:34:09.255 INFO:tasks.cephadm:Setting up client nodes... 2024-04-23T18:34:09.255 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-23T18:34:09.527 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:09.797 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:09 smithi132 ceph-mon[29178]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T18:34:09.797 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:09 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1817613016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:09.797 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:09 smithi132 ceph-mon[29178]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:34:09.797 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:09 smithi132 ceph-mon[29178]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:34:10.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:09 smithi148 ceph-mon[32311]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T18:34:10.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:09 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1817613016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:10.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:09 smithi148 ceph-mon[32311]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:34:10.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:09 smithi148 ceph-mon[32311]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:34:10.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:09 smithi157 ceph-mon[32580]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T18:34:10.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:09 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1817613016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:10.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:09 smithi157 ceph-mon[32580]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:34:10.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:09 smithi157 ceph-mon[32580]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:34:10.514 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:34:10 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b[62525]: 2024-04-23T18:34:10.186+0000 7f3555644700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-23T18:34:10.850 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:10 smithi132 ceph-mon[29178]: pgmap v294: 97 pgs: 97 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5.9 KiB/s wr, 22 op/s 2024-04-23T18:34:11.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:10 smithi148 ceph-mon[32311]: pgmap v294: 97 pgs: 97 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5.9 KiB/s wr, 22 op/s 2024-04-23T18:34:11.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:10 smithi157 ceph-mon[32580]: pgmap v294: 97 pgs: 97 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 5.9 KiB/s wr, 22 op/s 2024-04-23T18:34:11.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.249+0000 7fc05bc06700 1 -- 172.21.15.132:0/2634406729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054101520 msgr2=0x7fc05410b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.249+0000 7fc05bc06700 1 --2- 172.21.15.132:0/2634406729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054101520 0x7fc05410b0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00b0d0 tx=0x7fc04c00b3e0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.249+0000 7fc05bc06700 1 -- 172.21.15.132:0/2634406729 shutdown_connections 2024-04-23T18:34:11.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.249+0000 7fc05bc06700 1 --2- 172.21.15.132:0/2634406729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054101520 0x7fc05410b0d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.249+0000 7fc05bc06700 1 --2- 172.21.15.132:0/2634406729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0541040e0 0x7fc054104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.249+0000 7fc05bc06700 1 --2- 172.21.15.132:0/2634406729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc054100b60 0x7fc054100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.249+0000 7fc05bc06700 1 -- 172.21.15.132:0/2634406729 >> 172.21.15.132:0/2634406729 conn(0x7fc0540fc720 msgr2=0x7fc0540feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:11.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.249+0000 7fc05bc06700 1 -- 172.21.15.132:0/2634406729 shutdown_connections 2024-04-23T18:34:11.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.249+0000 7fc05bc06700 1 -- 172.21.15.132:0/2634406729 wait complete. 2024-04-23T18:34:11.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.249+0000 7fc05bc06700 1 Processor -- start 2024-04-23T18:34:11.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc05bc06700 1 -- start start 2024-04-23T18:34:11.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc05bc06700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc054100b60 0x7fc0541037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc05bc06700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054101520 0x7fc054101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc05bc06700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0541040e0 0x7fc054102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc05bc06700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc05410e570 con 0x7fc054101520 2024-04-23T18:34:11.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc05bc06700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc05410e3f0 con 0x7fc054100b60 2024-04-23T18:34:11.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc05bc06700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc05410e6f0 con 0x7fc0541040e0 2024-04-23T18:34:11.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc0591a1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054101520 0x7fc054101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc0591a1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054101520 0x7fc054101e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37274/0 (socket says 172.21.15.132:37274) 2024-04-23T18:34:11.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc0591a1700 1 -- 172.21.15.132:0/2130287930 learned_addr learned my addr 172.21.15.132:0/2130287930 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:11.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc05a1a3700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0541040e0 0x7fc054102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc0599a2700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc054100b60 0x7fc0541037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc0591a1700 1 -- 172.21.15.132:0/2130287930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0541040e0 msgr2=0x7fc054102370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc0591a1700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0541040e0 0x7fc054102370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc0591a1700 1 -- 172.21.15.132:0/2130287930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc054100b60 msgr2=0x7fc0541037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc0591a1700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc054100b60 0x7fc0541037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc0591a1700 1 -- 172.21.15.132:0/2130287930 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc054102c20 con 0x7fc054101520 2024-04-23T18:34:11.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc05a1a3700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0541040e0 0x7fc054102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:11.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc0599a2700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc054100b60 0x7fc0541037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:11.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.250+0000 7fc0591a1700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054101520 0x7fc054101e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc04400c990 tx=0x7fc04400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:11.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.251+0000 7fc04affd700 1 -- 172.21.15.132:0/2130287930 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc044007b00 con 0x7fc054101520 2024-04-23T18:34:11.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.251+0000 7fc05bc06700 1 -- 172.21.15.132:0/2130287930 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0541a2b80 con 0x7fc054101520 2024-04-23T18:34:11.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.251+0000 7fc04affd700 1 -- 172.21.15.132:0/2130287930 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc044007ca0 con 0x7fc054101520 2024-04-23T18:34:11.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.251+0000 7fc04affd700 1 -- 172.21.15.132:0/2130287930 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc044018660 con 0x7fc054101520 2024-04-23T18:34:11.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.251+0000 7fc05bc06700 1 -- 172.21.15.132:0/2130287930 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0541a30c0 con 0x7fc054101520 2024-04-23T18:34:11.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.252+0000 7fc05bc06700 1 -- 172.21.15.132:0/2130287930 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc054050f80 con 0x7fc054101520 2024-04-23T18:34:11.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.253+0000 7fc04affd700 1 -- 172.21.15.132:0/2130287930 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc044004030 con 0x7fc054101520 2024-04-23T18:34:11.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.253+0000 7fc04affd700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc0400a3d80 0x7fc0400a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.253+0000 7fc0599a2700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc0400a3d80 0x7fc0400a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.253+0000 7fc04affd700 1 -- 172.21.15.132:0/2130287930 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc0440d0570 con 0x7fc054101520 2024-04-23T18:34:11.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.254+0000 7fc0599a2700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc0400a3d80 0x7fc0400a6240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc050005950 tx=0x7fc0500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:11.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.257+0000 7fc04affd700 1 -- 172.21.15.132:0/2130287930 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc044014030 con 0x7fc054101520 2024-04-23T18:34:11.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.540+0000 7fc05bc06700 1 -- 172.21.15.132:0/2130287930 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc054063650 con 0x7fc054101520 2024-04-23T18:34:11.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.542+0000 7fc04affd700 1 -- 172.21.15.132:0/2130287930 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc04409cf30 con 0x7fc054101520 2024-04-23T18:34:11.534 INFO:teuthology.orchestra.run.smithi132.stdout:[client.0] 2024-04-23T18:34:11.534 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAj/ydm8ipFIBAA1vFqI8WLjQqcOH5eJSo68Q== 2024-04-23T18:34:11.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.544+0000 7fc05bc06700 1 -- 172.21.15.132:0/2130287930 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc0400a3d80 msgr2=0x7fc0400a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.544+0000 7fc05bc06700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc0400a3d80 0x7fc0400a6240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc050005950 tx=0x7fc0500058e0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.544+0000 7fc05bc06700 1 -- 172.21.15.132:0/2130287930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054101520 msgr2=0x7fc054101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.544+0000 7fc05bc06700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054101520 0x7fc054101e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc04400c990 tx=0x7fc04400cd50 comp rx=0 tx=0).stop 2024-04-23T18:34:11.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.545+0000 7fc05bc06700 1 -- 172.21.15.132:0/2130287930 shutdown_connections 2024-04-23T18:34:11.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.545+0000 7fc05bc06700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0541040e0 0x7fc054102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.545+0000 7fc05bc06700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc054100b60 0x7fc0541037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.545+0000 7fc05bc06700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc0400a3d80 0x7fc0400a6240 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.545+0000 7fc05bc06700 1 --2- 172.21.15.132:0/2130287930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054101520 0x7fc054101e30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.545+0000 7fc05bc06700 1 -- 172.21.15.132:0/2130287930 >> 172.21.15.132:0/2130287930 conn(0x7fc0540fc720 msgr2=0x7fc0541070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:11.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.545+0000 7fc05bc06700 1 -- 172.21.15.132:0/2130287930 shutdown_connections 2024-04-23T18:34:11.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:11.545+0000 7fc05bc06700 1 -- 172.21.15.132:0/2130287930 wait complete. 2024-04-23T18:34:11.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:11 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2130287930' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T18:34:11.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:11 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2130287930' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T18:34:12.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:11 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2130287930' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T18:34:12.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:11 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2130287930' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T18:34:12.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:11 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2130287930' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T18:34:12.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:11 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2130287930' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T18:34:12.260 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:34:12.260 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-23T18:34:12.260 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-23T18:34:12.295 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-23T18:34:12.295 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-23T18:34:12.296 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-23T18:34:12.576 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:12.831 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:12 smithi132 ceph-mon[29178]: pgmap v295: 97 pgs: 97 active+clean; 626 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-04-23T18:34:13.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:12 smithi148 ceph-mon[32311]: pgmap v295: 97 pgs: 97 active+clean; 626 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-04-23T18:34:13.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:12 smithi157 ceph-mon[32580]: pgmap v295: 97 pgs: 97 active+clean; 626 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-04-23T18:34:14.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.270+0000 7fc6423a2700 1 -- 172.21.15.132:0/825990582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c06bb30 msgr2=0x7fc63c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.270+0000 7fc6423a2700 1 --2- 172.21.15.132:0/825990582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c06bb30 0x7fc63c10f900 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc63400b0a0 tx=0x7fc63400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.270+0000 7fc6423a2700 1 -- 172.21.15.132:0/825990582 shutdown_connections 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.270+0000 7fc6423a2700 1 --2- 172.21.15.132:0/825990582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c06bb30 0x7fc63c10f900 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.270+0000 7fc6423a2700 1 --2- 172.21.15.132:0/825990582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc63c108a30 0x7fc63c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.270+0000 7fc6423a2700 1 --2- 172.21.15.132:0/825990582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc63c06b190 0x7fc63c06b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.270+0000 7fc6423a2700 1 -- 172.21.15.132:0/825990582 >> 172.21.15.132:0/825990582 conn(0x7fc63c078910 msgr2=0x7fc63c078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.270+0000 7fc6423a2700 1 -- 172.21.15.132:0/825990582 shutdown_connections 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.270+0000 7fc6423a2700 1 -- 172.21.15.132:0/825990582 wait complete. 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc6423a2700 1 Processor -- start 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc6423a2700 1 -- start start 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc6423a2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc63c06b190 0x7fc63c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc6423a2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc63c06bb30 0x7fc63c199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc6423a2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c108a30 0x7fc63c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc6423a2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c112bd0 con 0x7fc63c108a30 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc6423a2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c112a50 con 0x7fc63c06bb30 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc6423a2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c112d50 con 0x7fc63c06b190 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc64093f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c108a30 0x7fc63c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc64093f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c108a30 0x7fc63c19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37288/0 (socket says 172.21.15.132:37288) 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc64093f700 1 -- 172.21.15.132:0/710736280 learned_addr learned my addr 172.21.15.132:0/710736280 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc63b7fe700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc63c06bb30 0x7fc63c199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc63bfff700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc63c06b190 0x7fc63c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc64093f700 1 -- 172.21.15.132:0/710736280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc63c06b190 msgr2=0x7fc63c1990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc64093f700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc63c06b190 0x7fc63c1990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc64093f700 1 -- 172.21.15.132:0/710736280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc63c06bb30 msgr2=0x7fc63c199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc64093f700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc63c06bb30 0x7fc63c199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc64093f700 1 -- 172.21.15.132:0/710736280 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c19e0d0 con 0x7fc63c108a30 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.271+0000 7fc63bfff700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc63c06b190 0x7fc63c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.272+0000 7fc64093f700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c108a30 0x7fc63c19d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6340062a0 tx=0x7fc634004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.272+0000 7fc6397fa700 1 -- 172.21.15.132:0/710736280 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634021070 con 0x7fc63c108a30 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.272+0000 7fc6423a2700 1 -- 172.21.15.132:0/710736280 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63c19e360 con 0x7fc63c108a30 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.272+0000 7fc6397fa700 1 -- 172.21.15.132:0/710736280 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc634007c80 con 0x7fc63c108a30 2024-04-23T18:34:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.272+0000 7fc6423a2700 1 -- 172.21.15.132:0/710736280 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63c1a5c40 con 0x7fc63c108a30 2024-04-23T18:34:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.272+0000 7fc6397fa700 1 -- 172.21.15.132:0/710736280 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63401d450 con 0x7fc63c108a30 2024-04-23T18:34:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.273+0000 7fc6423a2700 1 -- 172.21.15.132:0/710736280 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc63c069730 con 0x7fc63c108a30 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.274+0000 7fc6397fa700 1 -- 172.21.15.132:0/710736280 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc634004020 con 0x7fc63c108a30 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.274+0000 7fc6397fa700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc6240a3cd0 0x7fc6240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.274+0000 7fc6397fa700 1 -- 172.21.15.132:0/710736280 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc63400f020 con 0x7fc63c108a30 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.274+0000 7fc63bfff700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc6240a3cd0 0x7fc6240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.275+0000 7fc63bfff700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc6240a3cd0 0x7fc6240a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc62c005fd0 tx=0x7fc62c005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:14.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.278+0000 7fc6397fa700 1 -- 172.21.15.132:0/710736280 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc63402a020 con 0x7fc63c108a30 2024-04-23T18:34:14.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.533+0000 7fc6423a2700 1 -- 172.21.15.132:0/710736280 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc63c19a420 con 0x7fc63c108a30 2024-04-23T18:34:14.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.537+0000 7fc6397fa700 1 -- 172.21.15.132:0/710736280 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+265003 (secure 0 0 0) 0x7fc63409e960 con 0x7fc63c108a30 2024-04-23T18:34:14.534 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:34:14.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.540+0000 7fc6423a2700 1 -- 172.21.15.132:0/710736280 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc6240a3cd0 msgr2=0x7fc6240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.540+0000 7fc6423a2700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc6240a3cd0 0x7fc6240a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc62c005fd0 tx=0x7fc62c005d40 comp rx=0 tx=0).stop 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.540+0000 7fc6423a2700 1 -- 172.21.15.132:0/710736280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c108a30 msgr2=0x7fc63c19d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.540+0000 7fc6423a2700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c108a30 0x7fc63c19d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6340062a0 tx=0x7fc634004e20 comp rx=0 tx=0).stop 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.540+0000 7fc6423a2700 1 -- 172.21.15.132:0/710736280 shutdown_connections 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.541+0000 7fc6423a2700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc6240a3cd0 0x7fc6240a6190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.541+0000 7fc6423a2700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c108a30 0x7fc63c19d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.541+0000 7fc6423a2700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc63c06b190 0x7fc63c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.541+0000 7fc6423a2700 1 --2- 172.21.15.132:0/710736280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc63c06bb30 0x7fc63c199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.541+0000 7fc6423a2700 1 -- 172.21.15.132:0/710736280 >> 172.21.15.132:0/710736280 conn(0x7fc63c078910 msgr2=0x7fc63c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.541+0000 7fc6423a2700 1 -- 172.21.15.132:0/710736280 shutdown_connections 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:14.541+0000 7fc6423a2700 1 -- 172.21.15.132:0/710736280 wait complete. 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":636352941},{"type":"v1","addr":"172.21.15.132:6801","nonce":636352941}]},"active_addr":"172.21.15.132:6801/636352941","active_change":"2024-04-23T18:26:54.042511+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.132:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":899407689}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":2124409853}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":1857586348}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":3095429281}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":2473422092}]}]} 2024-04-23T18:34:15.227 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-23T18:34:15.227 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-23T18:34:15.227 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T18:34:15.466 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:15.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:15 smithi148 ceph-mon[32311]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.4 KiB/s wr, 25 op/s 2024-04-23T18:34:15.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:15 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/710736280' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T18:34:15.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:15 smithi157 ceph-mon[32580]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.4 KiB/s wr, 25 op/s 2024-04-23T18:34:15.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:15 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/710736280' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T18:34:15.589 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:15 smithi132 ceph-mon[29178]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.4 KiB/s wr, 25 op/s 2024-04-23T18:34:15.589 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:15 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/710736280' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T18:34:17.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.118+0000 7fa5d5152700 1 -- 172.21.15.132:0/2797805265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5d01034b0 msgr2=0x7fa5d010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.118+0000 7fa5d5152700 1 --2- 172.21.15.132:0/2797805265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5d01034b0 0x7fa5d010f8e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800b0a0 tx=0x7fa5c800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.118+0000 7fa5d5152700 1 -- 172.21.15.132:0/2797805265 shutdown_connections 2024-04-23T18:34:17.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.118+0000 7fa5d5152700 1 --2- 172.21.15.132:0/2797805265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5d01034b0 0x7fa5d010f8e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.118+0000 7fa5d5152700 1 --2- 172.21.15.132:0/2797805265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5d0108a80 0x7fa5d0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.118+0000 7fa5d5152700 1 --2- 172.21.15.132:0/2797805265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5d0102af0 0x7fa5d0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.118+0000 7fa5d5152700 1 -- 172.21.15.132:0/2797805265 >> 172.21.15.132:0/2797805265 conn(0x7fa5d0079b20 msgr2=0x7fa5d0079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:17.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.118+0000 7fa5d5152700 1 -- 172.21.15.132:0/2797805265 shutdown_connections 2024-04-23T18:34:17.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.118+0000 7fa5d5152700 1 -- 172.21.15.132:0/2797805265 wait complete. 2024-04-23T18:34:17.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.119+0000 7fa5d5152700 1 Processor -- start 2024-04-23T18:34:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.119+0000 7fa5d5152700 1 -- start start 2024-04-23T18:34:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5d5152700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5d0102af0 0x7fa5d0199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5d5152700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5d01034b0 0x7fa5d0199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5d5152700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5d0108a80 0x7fa5d019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5d5152700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0112d10 con 0x7fa5d0102af0 2024-04-23T18:34:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5d5152700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0112b90 con 0x7fa5d01034b0 2024-04-23T18:34:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5d5152700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0112e90 con 0x7fa5d0108a80 2024-04-23T18:34:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5ced9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5d0102af0 0x7fa5d0199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5ced9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5d0102af0 0x7fa5d0199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37300/0 (socket says 172.21.15.132:37300) 2024-04-23T18:34:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5ced9d700 1 -- 172.21.15.132:0/2409264835 learned_addr learned my addr 172.21.15.132:0/2409264835 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5cf59e700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5d0108a80 0x7fa5d019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5ce59c700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5d01034b0 0x7fa5d0199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5ced9d700 1 -- 172.21.15.132:0/2409264835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5d0108a80 msgr2=0x7fa5d019dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5ced9d700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5d0108a80 0x7fa5d019dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5ced9d700 1 -- 172.21.15.132:0/2409264835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5d01034b0 msgr2=0x7fa5d0199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5ced9d700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5d01034b0 0x7fa5d0199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5ced9d700 1 -- 172.21.15.132:0/2409264835 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d019e350 con 0x7fa5d0102af0 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5ce59c700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5d01034b0 0x7fa5d0199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.120+0000 7fa5ced9d700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5d0102af0 0x7fa5d0199340 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000b530 tx=0x7fa5c000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.121+0000 7fa5b7fff700 1 -- 172.21.15.132:0/2409264835 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0015020 con 0x7fa5d0102af0 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.121+0000 7fa5d5152700 1 -- 172.21.15.132:0/2409264835 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d019e640 con 0x7fa5d0102af0 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.121+0000 7fa5b7fff700 1 -- 172.21.15.132:0/2409264835 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5c0011ce0 con 0x7fa5d0102af0 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.121+0000 7fa5b7fff700 1 -- 172.21.15.132:0/2409264835 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0010e40 con 0x7fa5d0102af0 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.121+0000 7fa5d5152700 1 -- 172.21.15.132:0/2409264835 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d01a5ec0 con 0x7fa5d0102af0 2024-04-23T18:34:17.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.122+0000 7fa5d5152700 1 -- 172.21.15.132:0/2409264835 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d0069730 con 0x7fa5d0102af0 2024-04-23T18:34:17.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.123+0000 7fa5b7fff700 1 -- 172.21.15.132:0/2409264835 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa5c0018020 con 0x7fa5d0102af0 2024-04-23T18:34:17.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.123+0000 7fa5b7fff700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa5b80a3cd0 0x7fa5b80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:17.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.123+0000 7fa5ce59c700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa5b80a3cd0 0x7fa5b80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:17.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.123+0000 7fa5b7fff700 1 -- 172.21.15.132:0/2409264835 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa5c00cfb60 con 0x7fa5d0102af0 2024-04-23T18:34:17.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.124+0000 7fa5ce59c700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa5b80a3cd0 0x7fa5b80a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5d007a250 tx=0x7fa5c4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:17.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.128+0000 7fa5b7fff700 1 -- 172.21.15.132:0/2409264835 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5c001f020 con 0x7fa5d0102af0 2024-04-23T18:34:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.334+0000 7fa5d5152700 1 -- 172.21.15.132:0/2409264835 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5d019a6a0 con 0x7fa5d0102af0 2024-04-23T18:34:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.336+0000 7fa5b7fff700 1 -- 172.21.15.132:0/2409264835 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22301 (secure 0 0 0) 0x7fa5c009c6a0 con 0x7fa5d0102af0 2024-04-23T18:34:17.328 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:34:17.328 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":86,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:34:02.205993+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:34:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.338+0000 7fa5d5152700 1 -- 172.21.15.132:0/2409264835 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa5b80a3cd0 msgr2=0x7fa5b80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.338+0000 7fa5d5152700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa5b80a3cd0 0x7fa5b80a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5d007a250 tx=0x7fa5c4008040 comp rx=0 tx=0).stop 2024-04-23T18:34:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.338+0000 7fa5d5152700 1 -- 172.21.15.132:0/2409264835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5d0102af0 msgr2=0x7fa5d0199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.338+0000 7fa5d5152700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5d0102af0 0x7fa5d0199340 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000b530 tx=0x7fa5c000b840 comp rx=0 tx=0).stop 2024-04-23T18:34:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.338+0000 7fa5d5152700 1 -- 172.21.15.132:0/2409264835 shutdown_connections 2024-04-23T18:34:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.338+0000 7fa5d5152700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5d0108a80 0x7fa5d019dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.338+0000 7fa5d5152700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa5b80a3cd0 0x7fa5b80a6190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.338+0000 7fa5d5152700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5d0102af0 0x7fa5d0199340 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.338+0000 7fa5d5152700 1 --2- 172.21.15.132:0/2409264835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5d01034b0 0x7fa5d0199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.338+0000 7fa5d5152700 1 -- 172.21.15.132:0/2409264835 >> 172.21.15.132:0/2409264835 conn(0x7fa5d0079b20 msgr2=0x7fa5d0101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.338+0000 7fa5d5152700 1 -- 172.21.15.132:0/2409264835 shutdown_connections 2024-04-23T18:34:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:17.338+0000 7fa5d5152700 1 -- 172.21.15.132:0/2409264835 wait complete. 2024-04-23T18:34:17.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:17 smithi148 ceph-mon[32311]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-04-23T18:34:17.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:17 smithi157 ceph-mon[32580]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-04-23T18:34:17.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:17 smithi132 ceph-mon[29178]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-04-23T18:34:18.077 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-23T18:34:18.077 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T18:34:18.317 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:18 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2409264835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:18.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:18 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2409264835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:18.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:18 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2409264835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:19.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:19 smithi148 ceph-mon[32311]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 22 op/s 2024-04-23T18:34:19.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:19 smithi132 ceph-mon[29178]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 22 op/s 2024-04-23T18:34:19.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:19 smithi157 ceph-mon[32580]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 22 op/s 2024-04-23T18:34:20.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.290+0000 7f4bfa157700 1 -- 172.21.15.132:0/2992082710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf41034b0 msgr2=0x7f4bf410f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:20.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.290+0000 7f4bfa157700 1 --2- 172.21.15.132:0/2992082710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf41034b0 0x7f4bf410f8e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00b0a0 tx=0x7f4bec00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:34:20.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.290+0000 7f4bfa157700 1 -- 172.21.15.132:0/2992082710 shutdown_connections 2024-04-23T18:34:20.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.290+0000 7f4bfa157700 1 --2- 172.21.15.132:0/2992082710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf41034b0 0x7f4bf410f8e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:20.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.290+0000 7f4bfa157700 1 --2- 172.21.15.132:0/2992082710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf4108a80 0x7f4bf4108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:20.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.290+0000 7f4bfa157700 1 --2- 172.21.15.132:0/2992082710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bf4102af0 0x7f4bf4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:20.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.290+0000 7f4bfa157700 1 -- 172.21.15.132:0/2992082710 >> 172.21.15.132:0/2992082710 conn(0x7f4bf4079ad0 msgr2=0x7f4bf4079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:20.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.290+0000 7f4bfa157700 1 -- 172.21.15.132:0/2992082710 shutdown_connections 2024-04-23T18:34:20.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.290+0000 7f4bfa157700 1 -- 172.21.15.132:0/2992082710 wait complete. 2024-04-23T18:34:20.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.290+0000 7f4bfa157700 1 Processor -- start 2024-04-23T18:34:20.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bfa157700 1 -- start start 2024-04-23T18:34:20.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bfa157700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bf4102af0 0x7f4bf4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:20.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bfa157700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf41034b0 0x7f4bf4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:20.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bfa157700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf4108a80 0x7f4bf419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:20.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bfa157700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4112d10 con 0x7f4bf41034b0 2024-04-23T18:34:20.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bfa157700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4112b90 con 0x7f4bf4108a80 2024-04-23T18:34:20.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bfa157700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4112e90 con 0x7f4bf4102af0 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bf2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf41034b0 0x7f4bf4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bf2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf41034b0 0x7f4bf4199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40912/0 (socket says 172.21.15.132:40912) 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bf2ffd700 1 -- 172.21.15.132:0/1743927717 learned_addr learned my addr 172.21.15.132:0/1743927717 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bf37fe700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bf4102af0 0x7f4bf4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bf3fff700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf4108a80 0x7f4bf419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bf2ffd700 1 -- 172.21.15.132:0/1743927717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bf4102af0 msgr2=0x7f4bf4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bf2ffd700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bf4102af0 0x7f4bf4199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bf2ffd700 1 -- 172.21.15.132:0/1743927717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf4108a80 msgr2=0x7f4bf419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bf2ffd700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf4108a80 0x7f4bf419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bf2ffd700 1 -- 172.21.15.132:0/1743927717 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf419e200 con 0x7f4bf41034b0 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.291+0000 7f4bf2ffd700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf41034b0 0x7f4bf4199780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4be400b530 tx=0x7f4be400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.292+0000 7f4bf0ff9700 1 -- 172.21.15.132:0/1743927717 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4015020 con 0x7f4bf41034b0 2024-04-23T18:34:20.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.292+0000 7f4bfa157700 1 -- 172.21.15.132:0/1743927717 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf419e4f0 con 0x7f4bf41034b0 2024-04-23T18:34:20.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.292+0000 7f4bf0ff9700 1 -- 172.21.15.132:0/1743927717 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4be4011ce0 con 0x7f4bf41034b0 2024-04-23T18:34:20.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.292+0000 7f4bf0ff9700 1 -- 172.21.15.132:0/1743927717 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4010e40 con 0x7f4bf41034b0 2024-04-23T18:34:20.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.292+0000 7f4bfa157700 1 -- 172.21.15.132:0/1743927717 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf41a5d70 con 0x7f4bf41034b0 2024-04-23T18:34:20.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.293+0000 7f4bfa157700 1 -- 172.21.15.132:0/1743927717 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf4069730 con 0x7f4bf41034b0 2024-04-23T18:34:20.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.294+0000 7f4bf0ff9700 1 -- 172.21.15.132:0/1743927717 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4be4018020 con 0x7f4bf41034b0 2024-04-23T18:34:20.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.295+0000 7f4bf0ff9700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4bdc0a3cd0 0x7f4bdc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:20.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.295+0000 7f4bf37fe700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4bdc0a3cd0 0x7f4bdc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:20.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.295+0000 7f4bf0ff9700 1 -- 172.21.15.132:0/1743927717 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4be40d0050 con 0x7f4bf41034b0 2024-04-23T18:34:20.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.295+0000 7f4bf37fe700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4bdc0a3cd0 0x7f4bdc0a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009a10 tx=0x7f4be8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:20.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.298+0000 7f4bf0ff9700 1 -- 172.21.15.132:0/1743927717 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4be401f020 con 0x7f4bf41034b0 2024-04-23T18:34:20.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.504+0000 7f4bfa157700 1 -- 172.21.15.132:0/1743927717 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bf402df80 con 0x7f4bf41034b0 2024-04-23T18:34:20.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.506+0000 7f4bf0ff9700 1 -- 172.21.15.132:0/1743927717 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22301 (secure 0 0 0) 0x7f4be409cb90 con 0x7f4bf41034b0 2024-04-23T18:34:20.498 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:34:20.498 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":86,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:34:02.205993+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:34:20.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.508+0000 7f4bfa157700 1 -- 172.21.15.132:0/1743927717 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4bdc0a3cd0 msgr2=0x7f4bdc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:20.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.508+0000 7f4bfa157700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4bdc0a3cd0 0x7f4bdc0a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009a10 tx=0x7f4be8009190 comp rx=0 tx=0).stop 2024-04-23T18:34:20.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.508+0000 7f4bfa157700 1 -- 172.21.15.132:0/1743927717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf41034b0 msgr2=0x7f4bf4199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:20.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.508+0000 7f4bfa157700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf41034b0 0x7f4bf4199780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4be400b530 tx=0x7f4be400b840 comp rx=0 tx=0).stop 2024-04-23T18:34:20.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.508+0000 7f4bfa157700 1 -- 172.21.15.132:0/1743927717 shutdown_connections 2024-04-23T18:34:20.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.508+0000 7f4bfa157700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf4108a80 0x7f4bf419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:20.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.508+0000 7f4bfa157700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bf4102af0 0x7f4bf4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:20.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.508+0000 7f4bfa157700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4bdc0a3cd0 0x7f4bdc0a6190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:20.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.508+0000 7f4bfa157700 1 --2- 172.21.15.132:0/1743927717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf41034b0 0x7f4bf4199780 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:20.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.508+0000 7f4bfa157700 1 -- 172.21.15.132:0/1743927717 >> 172.21.15.132:0/1743927717 conn(0x7f4bf4079ad0 msgr2=0x7f4bf41017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:20.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.508+0000 7f4bfa157700 1 -- 172.21.15.132:0/1743927717 shutdown_connections 2024-04-23T18:34:20.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:20.508+0000 7f4bfa157700 1 -- 172.21.15.132:0/1743927717 wait complete. 2024-04-23T18:34:21.266 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-23T18:34:21.266 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-23T18:34:21.267 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-23T18:34:21.267 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-23T18:34:21.267 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-23T18:34:21.268 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-23T18:34:21.268 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-23T18:34:21.269 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-23T18:34:21.269 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-23T18:34:21.269 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-23T18:34:21.270 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-23T18:34:21.270 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-23T18:34:21.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:21 smithi132 ceph-mon[29178]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 KiB/s wr, 18 op/s 2024-04-23T18:34:21.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:21 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1743927717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:21.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:21 smithi148 ceph-mon[32311]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 KiB/s wr, 18 op/s 2024-04-23T18:34:21.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:21 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1743927717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:21.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:21 smithi157 ceph-mon[32580]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 KiB/s wr, 18 op/s 2024-04-23T18:34:21.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:21 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1743927717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:21.949 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:21.949 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:21.949 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:21.951 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:21.955 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:21.962 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:21.963 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:22.004 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:23.458 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:23 smithi132 ceph-mon[29178]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-04-23T18:34:23.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:23 smithi148 ceph-mon[32311]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-04-23T18:34:23.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:23 smithi157 ceph-mon[32580]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-04-23T18:34:23.862 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:23.862 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:25.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:25 smithi148 ceph-mon[32311]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-23T18:34:25.524 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:25 smithi132 ceph-mon[29178]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-23T18:34:25.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:25 smithi157 ceph-mon[32580]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-23T18:34:25.818 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:27.465 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:27 smithi132 ceph-mon[29178]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-23T18:34:27.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:27 smithi148 ceph-mon[32311]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-23T18:34:27.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:27 smithi157 ceph-mon[32580]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-23T18:34:28.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.294+0000 7fdc0747d700 1 -- 172.21.15.132:0/447386111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00102330 msgr2=0x7fdc0010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:28.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.294+0000 7fdc0747d700 1 --2- 172.21.15.132:0/447386111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00102330 0x7fdc0010e7f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800b0a0 tx=0x7fdbf800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:34:28.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.294+0000 7fdc0747d700 1 -- 172.21.15.132:0/447386111 shutdown_connections 2024-04-23T18:34:28.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.294+0000 7fdc0747d700 1 --2- 172.21.15.132:0/447386111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00102330 0x7fdc0010e7f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.294+0000 7fdc0747d700 1 --2- 172.21.15.132:0/447386111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc00107920 0x7fdc00107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.294+0000 7fdc0747d700 1 --2- 172.21.15.132:0/447386111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc00101970 0x7fdc00101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.294+0000 7fdc0747d700 1 -- 172.21.15.132:0/447386111 >> 172.21.15.132:0/447386111 conn(0x7fdc000788f0 msgr2=0x7fdc00078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.294+0000 7fdc0747d700 1 -- 172.21.15.132:0/447386111 shutdown_connections 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.294+0000 7fdc0747d700 1 -- 172.21.15.132:0/447386111 wait complete. 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.294+0000 7fdc0747d700 1 Processor -- start 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.294+0000 7fdc0747d700 1 -- start start 2024-04-23T18:34:28.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc0747d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc00101970 0x7fdc001034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc0747d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc00102330 0x7fdc00103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc0747d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00107920 0x7fdc00103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc0747d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdc00111b60 con 0x7fdc00107920 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc0747d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc001119e0 con 0x7fdc00101970 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc0747d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc00111ce0 con 0x7fdc00102330 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc05a1a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00107920 0x7fdc00103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc05a1a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00107920 0x7fdc00103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40932/0 (socket says 172.21.15.132:40932) 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc05a1a700 1 -- 172.21.15.132:0/2225925344 learned_addr learned my addr 172.21.15.132:0/2225925344 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc04a18700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc00102330 0x7fdc00103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc05219700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc00101970 0x7fdc001034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc05a1a700 1 -- 172.21.15.132:0/2225925344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc00102330 msgr2=0x7fdc00103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc05a1a700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc00102330 0x7fdc00103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc05a1a700 1 -- 172.21.15.132:0/2225925344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc00101970 msgr2=0x7fdc001034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc05a1a700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc00101970 0x7fdc001034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc05a1a700 1 -- 172.21.15.132:0/2225925344 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc001a7550 con 0x7fdc00107920 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc05a1a700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00107920 0x7fdc00103f70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800b830 tx=0x7fdbf8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdbf67fc700 1 -- 172.21.15.132:0/2225925344 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8021070 con 0x7fdc00107920 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc0747d700 1 -- 172.21.15.132:0/2225925344 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc001a77e0 con 0x7fdc00107920 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdbf67fc700 1 -- 172.21.15.132:0/2225925344 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbf8007e30 con 0x7fdc00107920 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdbf67fc700 1 -- 172.21.15.132:0/2225925344 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf80047d0 con 0x7fdc00107920 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.295+0000 7fdc0747d700 1 -- 172.21.15.132:0/2225925344 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc001a7c40 con 0x7fdc00107920 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.296+0000 7fdc0747d700 1 -- 172.21.15.132:0/2225925344 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc00105900 con 0x7fdc00107920 2024-04-23T18:34:28.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.297+0000 7fdbf67fc700 1 -- 172.21.15.132:0/2225925344 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fdbf8007980 con 0x7fdc00107920 2024-04-23T18:34:28.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.298+0000 7fdbf67fc700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdbec0a3cd0 0x7fdbec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:28.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.298+0000 7fdc05219700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdbec0a3cd0 0x7fdbec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:28.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.298+0000 7fdbf67fc700 1 -- 172.21.15.132:0/2225925344 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdbf80d1860 con 0x7fdc00107920 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.298+0000 7fdbf67fc700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181] conn(0x7fdbec0ad4b0 0x7fdbec0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.298+0000 7fdbf67fc700 1 -- 172.21.15.132:0/2225925344 --> [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdbf801eec0 con 0x7fdbec0ad4b0 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.298+0000 7fdbf67fc700 1 -- 172.21.15.132:0/2225925344 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fdbf80d1d40 con 0x7fdc00107920 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.298+0000 7fdc05219700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdbec0a3cd0 0x7fdbec0a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009a90 tx=0x7fdbfc005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.298+0000 7fdc04a18700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181] conn(0x7fdbec0ad4b0 0x7fdbec0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.299+0000 7fdc04a18700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181] conn(0x7fdbec0ad4b0 0x7fdbec0af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.300+0000 7fdbf67fc700 1 -- 172.21.15.132:0/2225925344 <== osd.1 v2:172.21.15.148:6800/2532595181 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdbf801eec0 con 0x7fdbec0ad4b0 2024-04-23T18:34:28.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdc0747d700 1 -- 172.21.15.132:0/2225925344 --> [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdc00103060 con 0x7fdbec0ad4b0 2024-04-23T18:34:28.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdbf67fc700 1 -- 172.21.15.132:0/2225925344 <== osd.1 v2:172.21.15.148:6800/2532595181 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdc00103060 con 0x7fdbec0ad4b0 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdc0747d700 1 -- 172.21.15.132:0/2225925344 >> [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181] conn(0x7fdbec0ad4b0 msgr2=0x7fdbec0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdc0747d700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181] conn(0x7fdbec0ad4b0 0x7fdbec0af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdc0747d700 1 -- 172.21.15.132:0/2225925344 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdbec0a3cd0 msgr2=0x7fdbec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdc0747d700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdbec0a3cd0 0x7fdbec0a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009a90 tx=0x7fdbfc005ee0 comp rx=0 tx=0).stop 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdc0747d700 1 -- 172.21.15.132:0/2225925344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00107920 msgr2=0x7fdc00103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdc0747d700 1 --2- 172.21.15.132:0/2225925344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00107920 0x7fdc00103f70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800b830 tx=0x7fdbf8004ce0 comp rx=0 tx=0).stop 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdc05a1a700 1 -- 172.21.15.132:0/2225925344 reap_dead start 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdc0747d700 1 -- 172.21.15.132:0/2225925344 shutdown_connections 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdc0747d700 1 -- 172.21.15.132:0/2225925344 >> 172.21.15.132:0/2225925344 conn(0x7fdc000788f0 msgr2=0x7fdc00100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdc0747d700 1 -- 172.21.15.132:0/2225925344 shutdown_connections 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:28.317+0000 7fdc0747d700 1 -- 172.21.15.132:0/2225925344 wait complete. 2024-04-23T18:34:29.500 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:29 smithi132 ceph-mon[29178]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T18:34:29.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:29 smithi148 ceph-mon[32311]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T18:34:29.527 INFO:teuthology.orchestra.run.smithi132.stdout:55834574917 2024-04-23T18:34:29.527 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-23T18:34:29.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:29 smithi157 ceph-mon[32580]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T18:34:31.499 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:31 smithi132 ceph-mon[29178]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T18:34:31.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:31 smithi148 ceph-mon[32311]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T18:34:31.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:31 smithi157 ceph-mon[32580]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T18:34:33.355 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:33 smithi132 ceph-mon[29178]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-23T18:34:33.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:33 smithi148 ceph-mon[32311]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-23T18:34:33.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:33 smithi157 ceph-mon[32580]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-23T18:34:35.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:35 smithi132 ceph-mon[29178]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:34:35.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:35 smithi148 ceph-mon[32311]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:34:35.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:35 smithi157 ceph-mon[32580]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:34:35.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.682+0000 7fa735105700 1 -- 172.21.15.132:0/1408055608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa730108a30 msgr2=0x7fa730108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:35.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.682+0000 7fa735105700 1 --2- 172.21.15.132:0/1408055608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa730108a30 0x7fa730108e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa724009990 tx=0x7fa724009ca0 comp rx=0 tx=0).stop 2024-04-23T18:34:35.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.682+0000 7fa735105700 1 -- 172.21.15.132:0/1408055608 shutdown_connections 2024-04-23T18:34:35.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.682+0000 7fa735105700 1 --2- 172.21.15.132:0/1408055608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa73006b0f0 0x7fa73010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:35.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.682+0000 7fa735105700 1 --2- 172.21.15.132:0/1408055608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa730108a30 0x7fa730108e10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:35.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.682+0000 7fa735105700 1 --2- 172.21.15.132:0/1408055608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa73006a750 0x7fa73006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:35.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.682+0000 7fa735105700 1 -- 172.21.15.132:0/1408055608 >> 172.21.15.132:0/1408055608 conn(0x7fa7300788f0 msgr2=0x7fa730078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:35.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.682+0000 7fa735105700 1 -- 172.21.15.132:0/1408055608 shutdown_connections 2024-04-23T18:34:35.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.682+0000 7fa735105700 1 -- 172.21.15.132:0/1408055608 wait complete. 2024-04-23T18:34:35.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa735105700 1 Processor -- start 2024-04-23T18:34:35.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa735105700 1 -- start start 2024-04-23T18:34:35.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa735105700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa73006a750 0x7fa73019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:35.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa735105700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa73006b0f0 0x7fa73019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:35.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa735105700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa730108a30 0x7fa73019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:35.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa735105700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa730112c50 con 0x7fa730108a30 2024-04-23T18:34:35.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa735105700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa730112ad0 con 0x7fa73006b0f0 2024-04-23T18:34:35.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa735105700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa730112dd0 con 0x7fa73006a750 2024-04-23T18:34:35.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa72f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa730108a30 0x7fa73019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:35.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa72f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa730108a30 0x7fa73019fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34402/0 (socket says 172.21.15.132:34402) 2024-04-23T18:34:35.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa72f59e700 1 -- 172.21.15.132:0/3807249920 learned_addr learned my addr 172.21.15.132:0/3807249920 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:35.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa72ed9d700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa73006a750 0x7fa73019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:35.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa72e59c700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa73006b0f0 0x7fa73019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:35.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa72f59e700 1 -- 172.21.15.132:0/3807249920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa73006a750 msgr2=0x7fa73019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:35.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa72f59e700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa73006a750 0x7fa73019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:35.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa72f59e700 1 -- 172.21.15.132:0/3807249920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa73006b0f0 msgr2=0x7fa73019b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:35.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa72f59e700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa73006b0f0 0x7fa73019b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:35.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.683+0000 7fa72f59e700 1 -- 172.21.15.132:0/3807249920 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7301a03b0 con 0x7fa730108a30 2024-04-23T18:34:35.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.684+0000 7fa72f59e700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa730108a30 0x7fa73019fc70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa72800b970 tx=0x7fa72800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:35.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.684+0000 7fa717fff700 1 -- 172.21.15.132:0/3807249920 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728019070 con 0x7fa730108a30 2024-04-23T18:34:35.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.684+0000 7fa717fff700 1 -- 172.21.15.132:0/3807249920 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa72800cde0 con 0x7fa730108a30 2024-04-23T18:34:35.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.684+0000 7fa717fff700 1 -- 172.21.15.132:0/3807249920 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728007370 con 0x7fa730108a30 2024-04-23T18:34:35.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.684+0000 7fa735105700 1 -- 172.21.15.132:0/3807249920 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7301a06a0 con 0x7fa730108a30 2024-04-23T18:34:35.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.684+0000 7fa735105700 1 -- 172.21.15.132:0/3807249920 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7301a7f20 con 0x7fa730108a30 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.685+0000 7fa735105700 1 -- 172.21.15.132:0/3807249920 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa730100190 con 0x7fa730108a30 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.686+0000 7fa717fff700 1 -- 172.21.15.132:0/3807249920 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa72801c070 con 0x7fa730108a30 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.686+0000 7fa717fff700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa7180a3cd0 0x7fa7180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.686+0000 7fa72ed9d700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa7180a3cd0 0x7fa7180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.686+0000 7fa717fff700 1 -- 172.21.15.132:0/3807249920 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa7280d01c0 con 0x7fa730108a30 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.687+0000 7fa717fff700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445] conn(0x7fa7180ad4b0 0x7fa7180af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.687+0000 7fa717fff700 1 -- 172.21.15.132:0/3807249920 --> [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa72801ce50 con 0x7fa7180ad4b0 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.687+0000 7fa717fff700 1 -- 172.21.15.132:0/3807249920 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fa7280d06a0 con 0x7fa730108a30 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.687+0000 7fa72e59c700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445] conn(0x7fa7180ad4b0 0x7fa7180af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.687+0000 7fa72ed9d700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa7180a3cd0 0x7fa7180a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa72400b5c0 tx=0x7fa724005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.687+0000 7fa72e59c700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445] conn(0x7fa7180ad4b0 0x7fa7180af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.688+0000 7fa717fff700 1 -- 172.21.15.132:0/3807249920 <== osd.9 v2:172.21.15.132:6826/3598365445 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa72801ce50 con 0x7fa7180ad4b0 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.704+0000 7fa735105700 1 -- 172.21.15.132:0/3807249920 --> [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa730104170 con 0x7fa7180ad4b0 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.704+0000 7fa717fff700 1 -- 172.21.15.132:0/3807249920 <== osd.9 v2:172.21.15.132:6826/3598365445 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa730104170 con 0x7fa7180ad4b0 2024-04-23T18:34:35.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.704+0000 7fa735105700 1 -- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445] conn(0x7fa7180ad4b0 msgr2=0x7fa7180af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:35.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.704+0000 7fa735105700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445] conn(0x7fa7180ad4b0 0x7fa7180af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:35.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.704+0000 7fa735105700 1 -- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa7180a3cd0 msgr2=0x7fa7180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:35.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.704+0000 7fa735105700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa7180a3cd0 0x7fa7180a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa72400b5c0 tx=0x7fa724005960 comp rx=0 tx=0).stop 2024-04-23T18:34:35.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.704+0000 7fa735105700 1 -- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa730108a30 msgr2=0x7fa73019fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:35.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.704+0000 7fa735105700 1 --2- 172.21.15.132:0/3807249920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa730108a30 0x7fa73019fc70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa72800b970 tx=0x7fa72800bc80 comp rx=0 tx=0).stop 2024-04-23T18:34:35.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.704+0000 7fa72f59e700 1 -- 172.21.15.132:0/3807249920 reap_dead start 2024-04-23T18:34:35.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.705+0000 7fa735105700 1 -- 172.21.15.132:0/3807249920 shutdown_connections 2024-04-23T18:34:35.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.705+0000 7fa735105700 1 -- 172.21.15.132:0/3807249920 >> 172.21.15.132:0/3807249920 conn(0x7fa7300788f0 msgr2=0x7fa7300ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:35.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.705+0000 7fa735105700 1 -- 172.21.15.132:0/3807249920 shutdown_connections 2024-04-23T18:34:35.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:35.705+0000 7fa735105700 1 -- 172.21.15.132:0/3807249920 wait complete. 2024-04-23T18:34:36.997 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.997+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2593856797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf0108ab0 msgr2=0x7f3cf0108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:36.997 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.997+0000 7f3cf4b30700 1 --2- 172.21.15.132:0/2593856797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf0108ab0 0x7f3cf0108e90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00099c0 tx=0x7f3ce0009cd0 comp rx=0 tx=0).stop 2024-04-23T18:34:36.997 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.997+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2593856797 shutdown_connections 2024-04-23T18:34:36.997 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.997+0000 7f3cf4b30700 1 --2- 172.21.15.132:0/2593856797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf006bb50 0x7f3cf010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:36.997 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.997+0000 7f3cf4b30700 1 --2- 172.21.15.132:0/2593856797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf0108ab0 0x7f3cf0108e90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:36.997 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.997+0000 7f3cf4b30700 1 --2- 172.21.15.132:0/2593856797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf006b190 0x7f3cf006b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:36.997 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.997+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2593856797 >> 172.21.15.132:0/2593856797 conn(0x7f3cf0078830 msgr2=0x7f3cf0078c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.997+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2593856797 shutdown_connections 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.997+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2593856797 wait complete. 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3cf4b30700 1 Processor -- start 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3cf4b30700 1 -- start start 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3cf4b30700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf006b190 0x7f3cf01990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3cf4b30700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf006bb50 0x7f3cf0199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3cf4b30700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf0108ab0 0x7f3cf019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3cf4b30700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0112cc0 con 0x7f3cf006bb50 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3cf4b30700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0112b40 con 0x7f3cf0108ab0 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3cf4b30700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0112e40 con 0x7f3cf006b190 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3cedd9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf006bb50 0x7f3cf0199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3cedd9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf006bb50 0x7f3cf0199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34414/0 (socket says 172.21.15.132:34414) 2024-04-23T18:34:36.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3cedd9b700 1 -- 172.21.15.132:0/2821259333 learned_addr learned my addr 172.21.15.132:0/2821259333 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3ceed9d700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf0108ab0 0x7f3cf019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.998+0000 7f3cee59c700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf006b190 0x7f3cf01990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.999+0000 7f3cedd9b700 1 -- 172.21.15.132:0/2821259333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf006b190 msgr2=0x7f3cf01990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.999+0000 7f3cedd9b700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf006b190 0x7f3cf01990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.999+0000 7f3cedd9b700 1 -- 172.21.15.132:0/2821259333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf0108ab0 msgr2=0x7f3cf019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.999+0000 7f3cedd9b700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf0108ab0 0x7f3cf019d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.999+0000 7f3cedd9b700 1 -- 172.21.15.132:0/2821259333 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf019e0e0 con 0x7f3cf006bb50 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.999+0000 7f3cee59c700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf006b190 0x7f3cf01990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.999+0000 7f3cedd9b700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf006bb50 0x7f3cf0199610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400c9b0 tx=0x7f3ce400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.999+0000 7f3cdf7fe700 1 -- 172.21.15.132:0/2821259333 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4007b00 con 0x7f3cf006bb50 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.999+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2821259333 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf019e3d0 con 0x7f3cf006bb50 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.999+0000 7f3cdf7fe700 1 -- 172.21.15.132:0/2821259333 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ce4007ca0 con 0x7f3cf006bb50 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.999+0000 7f3cdf7fe700 1 -- 172.21.15.132:0/2821259333 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4018730 con 0x7f3cf006bb50 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:36.999+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2821259333 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf01a5c50 con 0x7f3cf006bb50 2024-04-23T18:34:36.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.000+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2821259333 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3cf00631d0 con 0x7f3cf006bb50 2024-04-23T18:34:37.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.001+0000 7f3cdf7fe700 1 -- 172.21.15.132:0/2821259333 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3ce400f450 con 0x7f3cf006bb50 2024-04-23T18:34:37.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.001+0000 7f3cdf7fe700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3cd80a3cd0 0x7f3cd80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:37.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.002+0000 7f3cee59c700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3cd80a3cd0 0x7f3cd80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:37.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.002+0000 7f3cdf7fe700 1 -- 172.21.15.132:0/2821259333 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3ce40d08f0 con 0x7f3cf006bb50 2024-04-23T18:34:37.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.002+0000 7f3cdf7fe700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857] conn(0x7f3cd80ad4b0 0x7f3cd80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:37.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.002+0000 7f3cdf7fe700 1 -- 172.21.15.132:0/2821259333 --> [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ce4004500 con 0x7f3cd80ad4b0 2024-04-23T18:34:37.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.002+0000 7f3cdf7fe700 1 -- 172.21.15.132:0/2821259333 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f3ce40d0dd0 con 0x7f3cf006bb50 2024-04-23T18:34:37.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.002+0000 7f3cee59c700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3cd80a3cd0 0x7f3cd80a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009750 tx=0x7f3ce0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:37.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.002+0000 7f3ceed9d700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857] conn(0x7f3cd80ad4b0 0x7f3cd80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:37.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.003+0000 7f3ceed9d700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857] conn(0x7f3cd80ad4b0 0x7f3cd80af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:37.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.003+0000 7f3cdf7fe700 1 -- 172.21.15.132:0/2821259333 <== osd.2 v2:172.21.15.157:6800/1299586857 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ce4004500 con 0x7f3cd80ad4b0 2024-04-23T18:34:37.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.019+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2821259333 --> [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3cf0063600 con 0x7f3cd80ad4b0 2024-04-23T18:34:37.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.020+0000 7f3cdf7fe700 1 -- 172.21.15.132:0/2821259333 <== osd.2 v2:172.21.15.157:6800/1299586857 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3cf0063600 con 0x7f3cd80ad4b0 2024-04-23T18:34:37.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.020+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2821259333 >> [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857] conn(0x7f3cd80ad4b0 msgr2=0x7f3cd80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:37.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.020+0000 7f3cf4b30700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857] conn(0x7f3cd80ad4b0 0x7f3cd80af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:37.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.020+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2821259333 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3cd80a3cd0 msgr2=0x7f3cd80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:37.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.020+0000 7f3cf4b30700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3cd80a3cd0 0x7f3cd80a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009750 tx=0x7f3ce0005fb0 comp rx=0 tx=0).stop 2024-04-23T18:34:37.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.020+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2821259333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf006bb50 msgr2=0x7f3cf0199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:37.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.020+0000 7f3cf4b30700 1 --2- 172.21.15.132:0/2821259333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf006bb50 0x7f3cf0199610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400c9b0 tx=0x7f3ce400cd70 comp rx=0 tx=0).stop 2024-04-23T18:34:37.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.020+0000 7f3ceed9d700 1 -- 172.21.15.132:0/2821259333 reap_dead start 2024-04-23T18:34:37.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.020+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2821259333 shutdown_connections 2024-04-23T18:34:37.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.020+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2821259333 >> 172.21.15.132:0/2821259333 conn(0x7f3cf0078830 msgr2=0x7f3cf00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:37.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.021+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2821259333 shutdown_connections 2024-04-23T18:34:37.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.021+0000 7f3cf4b30700 1 -- 172.21.15.132:0/2821259333 wait complete. 2024-04-23T18:34:37.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:37 smithi148 ceph-mon[32311]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:34:37.535 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:37 smithi132 ceph-mon[29178]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:34:37.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:37 smithi157 ceph-mon[32580]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:34:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.737+0000 7ff15d06e700 1 -- 172.21.15.132:0/2845830091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff158101970 msgr2=0x7ff158101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.737+0000 7ff15d06e700 1 --2- 172.21.15.132:0/2845830091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff158101970 0x7ff158101df0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009990 tx=0x7ff14c009ca0 comp rx=0 tx=0).stop 2024-04-23T18:34:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.737+0000 7ff15d06e700 1 -- 172.21.15.132:0/2845830091 shutdown_connections 2024-04-23T18:34:37.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.737+0000 7ff15d06e700 1 --2- 172.21.15.132:0/2845830091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff158102330 0x7ff15810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:37.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.737+0000 7ff15d06e700 1 --2- 172.21.15.132:0/2845830091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff158107920 0x7ff158107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:37.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.737+0000 7ff15d06e700 1 --2- 172.21.15.132:0/2845830091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff158101970 0x7ff158101df0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:37.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.737+0000 7ff15d06e700 1 -- 172.21.15.132:0/2845830091 >> 172.21.15.132:0/2845830091 conn(0x7ff1580788f0 msgr2=0x7ff158078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:37.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.737+0000 7ff15d06e700 1 -- 172.21.15.132:0/2845830091 shutdown_connections 2024-04-23T18:34:37.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.737+0000 7ff15d06e700 1 -- 172.21.15.132:0/2845830091 wait complete. 2024-04-23T18:34:37.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff15d06e700 1 Processor -- start 2024-04-23T18:34:37.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff15d06e700 1 -- start start 2024-04-23T18:34:37.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff15d06e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff158101970 0x7ff158197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:37.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff15d06e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff158102330 0x7ff158198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff15d06e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff158107920 0x7ff158198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff15d06e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff158111bc0 con 0x7ff158101970 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff15d06e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff158111a40 con 0x7ff158107920 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff15d06e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff158111d40 con 0x7ff158102330 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff156d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff158101970 0x7ff158197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff156d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff158101970 0x7ff158197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34424/0 (socket says 172.21.15.132:34424) 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff156d9d700 1 -- 172.21.15.132:0/856458926 learned_addr learned my addr 172.21.15.132:0/856458926 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff15759e700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff158107920 0x7ff158198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.738+0000 7ff15659c700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff158102330 0x7ff158198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.739+0000 7ff156d9d700 1 -- 172.21.15.132:0/856458926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff158102330 msgr2=0x7ff158198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.739+0000 7ff156d9d700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff158102330 0x7ff158198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.739+0000 7ff156d9d700 1 -- 172.21.15.132:0/856458926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff158107920 msgr2=0x7ff158198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.739+0000 7ff156d9d700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff158107920 0x7ff158198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:37.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.739+0000 7ff156d9d700 1 -- 172.21.15.132:0/856458926 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1581a7780 con 0x7ff158101970 2024-04-23T18:34:37.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.739+0000 7ff15759e700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff158107920 0x7ff158198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:37.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.739+0000 7ff156d9d700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff158101970 0x7ff158197ed0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff14800b530 tx=0x7ff14800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:37.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.739+0000 7ff13ffff700 1 -- 172.21.15.132:0/856458926 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148015020 con 0x7ff158101970 2024-04-23T18:34:37.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.739+0000 7ff13ffff700 1 -- 172.21.15.132:0/856458926 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff148011ce0 con 0x7ff158101970 2024-04-23T18:34:37.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.739+0000 7ff13ffff700 1 -- 172.21.15.132:0/856458926 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148010dd0 con 0x7ff158101970 2024-04-23T18:34:37.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.739+0000 7ff15d06e700 1 -- 172.21.15.132:0/856458926 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1581a7a70 con 0x7ff158101970 2024-04-23T18:34:37.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.739+0000 7ff15d06e700 1 -- 172.21.15.132:0/856458926 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1581a7f30 con 0x7ff158101970 2024-04-23T18:34:37.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.740+0000 7ff15d06e700 1 -- 172.21.15.132:0/856458926 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff158063150 con 0x7ff158101970 2024-04-23T18:34:37.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.741+0000 7ff13ffff700 1 -- 172.21.15.132:0/856458926 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff148018020 con 0x7ff158101970 2024-04-23T18:34:37.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.742+0000 7ff13ffff700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff1400a3d50 0x7ff1400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:37.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.742+0000 7ff15659c700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff1400a3d50 0x7ff1400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:37.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.742+0000 7ff13ffff700 1 -- 172.21.15.132:0/856458926 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff1480d0090 con 0x7ff158101970 2024-04-23T18:34:37.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.742+0000 7ff13ffff700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155] conn(0x7ff1400ad530 0x7ff1400af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:37.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.742+0000 7ff13ffff700 1 -- 172.21.15.132:0/856458926 --> [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff148018ec0 con 0x7ff1400ad530 2024-04-23T18:34:37.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.742+0000 7ff13ffff700 1 -- 172.21.15.132:0/856458926 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7ff1480d0570 con 0x7ff158101970 2024-04-23T18:34:37.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.742+0000 7ff15759e700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155] conn(0x7ff1400ad530 0x7ff1400af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:37.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.743+0000 7ff15659c700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff1400a3d50 0x7ff1400a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00b5c0 tx=0x7ff14c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:37.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.743+0000 7ff15759e700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155] conn(0x7ff1400ad530 0x7ff1400af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:37.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.743+0000 7ff13ffff700 1 -- 172.21.15.132:0/856458926 <== osd.3 v2:172.21.15.132:6810/2500771155 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff148018ec0 con 0x7ff1400ad530 2024-04-23T18:34:37.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.760+0000 7ff15d06e700 1 -- 172.21.15.132:0/856458926 --> [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff158063420 con 0x7ff1400ad530 2024-04-23T18:34:37.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.760+0000 7ff13ffff700 1 -- 172.21.15.132:0/856458926 <== osd.3 v2:172.21.15.132:6810/2500771155 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff158063420 con 0x7ff1400ad530 2024-04-23T18:34:37.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.760+0000 7ff15d06e700 1 -- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155] conn(0x7ff1400ad530 msgr2=0x7ff1400af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:37.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.760+0000 7ff15d06e700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155] conn(0x7ff1400ad530 0x7ff1400af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:37.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.760+0000 7ff15d06e700 1 -- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff1400a3d50 msgr2=0x7ff1400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:37.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.760+0000 7ff15d06e700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff1400a3d50 0x7ff1400a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00b5c0 tx=0x7ff14c005960 comp rx=0 tx=0).stop 2024-04-23T18:34:37.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.760+0000 7ff15d06e700 1 -- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff158101970 msgr2=0x7ff158197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:37.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.760+0000 7ff15d06e700 1 --2- 172.21.15.132:0/856458926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff158101970 0x7ff158197ed0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff14800b530 tx=0x7ff14800b840 comp rx=0 tx=0).stop 2024-04-23T18:34:37.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.760+0000 7ff15759e700 1 -- 172.21.15.132:0/856458926 reap_dead start 2024-04-23T18:34:37.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.760+0000 7ff15d06e700 1 -- 172.21.15.132:0/856458926 shutdown_connections 2024-04-23T18:34:37.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.760+0000 7ff15d06e700 1 -- 172.21.15.132:0/856458926 >> 172.21.15.132:0/856458926 conn(0x7ff1580788f0 msgr2=0x7ff1580ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:37.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.761+0000 7ff15d06e700 1 -- 172.21.15.132:0/856458926 shutdown_connections 2024-04-23T18:34:37.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:37.761+0000 7ff15d06e700 1 -- 172.21.15.132:0/856458926 wait complete. 2024-04-23T18:34:39.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:39 smithi132 ceph-mon[29178]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-23T18:34:39.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:39 smithi148 ceph-mon[32311]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-23T18:34:39.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:39 smithi157 ceph-mon[32580]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-23T18:34:39.745 INFO:teuthology.orchestra.run.smithi132.stdout:77309411395 2024-04-23T18:34:39.745 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-23T18:34:40.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.276+0000 7f24c8038700 1 -- 172.21.15.132:0/2037663316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c0101cf0 msgr2=0x7f24c0105d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:40.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.276+0000 7f24c8038700 1 --2- 172.21.15.132:0/2037663316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c0101cf0 0x7f24c0105d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f24b00099c0 tx=0x7f24b0009cd0 comp rx=0 tx=0).stop 2024-04-23T18:34:40.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.276+0000 7f24c8038700 1 -- 172.21.15.132:0/2037663316 shutdown_connections 2024-04-23T18:34:40.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.276+0000 7f24c8038700 1 --2- 172.21.15.132:0/2037663316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24c0106420 0x7f24c010dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:40.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.276+0000 7f24c8038700 1 --2- 172.21.15.132:0/2037663316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24c0101340 0x7f24c0101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:40.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.276+0000 7f24c8038700 1 --2- 172.21.15.132:0/2037663316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c0101cf0 0x7f24c0105d40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:40.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.276+0000 7f24c8038700 1 -- 172.21.15.132:0/2037663316 >> 172.21.15.132:0/2037663316 conn(0x7f24c0077f40 msgr2=0x7f24c00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:40.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.277+0000 7f24c8038700 1 -- 172.21.15.132:0/2037663316 shutdown_connections 2024-04-23T18:34:40.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.277+0000 7f24c8038700 1 -- 172.21.15.132:0/2037663316 wait complete. 2024-04-23T18:34:40.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.277+0000 7f24c8038700 1 Processor -- start 2024-04-23T18:34:40.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.277+0000 7f24c8038700 1 -- start start 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.277+0000 7f24c8038700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24c0101340 0x7f24c019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.277+0000 7f24c8038700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c0101cf0 0x7f24c019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.277+0000 7f24c8038700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24c0106420 0x7f24c01a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.277+0000 7f24c8038700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f24c0111ba0 con 0x7f24c0101cf0 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.277+0000 7f24c8038700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f24c0111a20 con 0x7f24c0106420 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.277+0000 7f24c8038700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f24c0111d20 con 0x7f24c0101340 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c55d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c0101cf0 0x7f24c019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c55d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c0101cf0 0x7f24c019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53004/0 (socket says 172.21.15.132:53004) 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c55d3700 1 -- 172.21.15.132:0/2539826066 learned_addr learned my addr 172.21.15.132:0/2539826066 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c65d5700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24c0106420 0x7f24c01a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c5dd4700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24c0101340 0x7f24c019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c55d3700 1 -- 172.21.15.132:0/2539826066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24c0101340 msgr2=0x7f24c019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:40.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c55d3700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24c0101340 0x7f24c019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:40.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c55d3700 1 -- 172.21.15.132:0/2539826066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24c0106420 msgr2=0x7f24c01a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:40.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c55d3700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24c0106420 0x7f24c01a50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:40.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c55d3700 1 -- 172.21.15.132:0/2539826066 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c01a5600 con 0x7f24c0101cf0 2024-04-23T18:34:40.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c55d3700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c0101cf0 0x7f24c019a580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009f50 tx=0x7f24b00037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:40.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24b6ffd700 1 -- 172.21.15.132:0/2539826066 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0027070 con 0x7f24c0101cf0 2024-04-23T18:34:40.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c8038700 1 -- 172.21.15.132:0/2539826066 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c01a57d0 con 0x7f24c0101cf0 2024-04-23T18:34:40.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24b6ffd700 1 -- 172.21.15.132:0/2539826066 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24b0004090 con 0x7f24c0101cf0 2024-04-23T18:34:40.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24b6ffd700 1 -- 172.21.15.132:0/2539826066 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0022e60 con 0x7f24c0101cf0 2024-04-23T18:34:40.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.278+0000 7f24c8038700 1 -- 172.21.15.132:0/2539826066 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c01a5bd0 con 0x7f24c0101cf0 2024-04-23T18:34:40.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.279+0000 7f24c8038700 1 -- 172.21.15.132:0/2539826066 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24c0063110 con 0x7f24c0101cf0 2024-04-23T18:34:40.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.280+0000 7f24b6ffd700 1 -- 172.21.15.132:0/2539826066 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f24b002a070 con 0x7f24c0101cf0 2024-04-23T18:34:40.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.281+0000 7f24b6ffd700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24ac0a3cd0 0x7f24ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:40.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.281+0000 7f24c5dd4700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24ac0a3cd0 0x7f24ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:40.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.281+0000 7f24b6ffd700 1 -- 172.21.15.132:0/2539826066 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f24b00d1930 con 0x7f24c0101cf0 2024-04-23T18:34:40.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.281+0000 7f24b6ffd700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413] conn(0x7f24ac0ad380 0x7f24ac0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:40.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.281+0000 7f24b6ffd700 1 -- 172.21.15.132:0/2539826066 --> [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24b000ce90 con 0x7f24ac0ad380 2024-04-23T18:34:40.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.281+0000 7f24b6ffd700 1 -- 172.21.15.132:0/2539826066 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f24b00d1e10 con 0x7f24c0101cf0 2024-04-23T18:34:40.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.281+0000 7f24c5dd4700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24ac0a3cd0 0x7f24ac0a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f24bc006fd0 tx=0x7f24bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:40.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.282+0000 7f24c65d5700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413] conn(0x7f24ac0ad380 0x7f24ac0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:40.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.282+0000 7f24c65d5700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413] conn(0x7f24ac0ad380 0x7f24ac0af7a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:40.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.283+0000 7f24b6ffd700 1 -- 172.21.15.132:0/2539826066 <== osd.7 v2:172.21.15.148:6816/199627413 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24b000ce90 con 0x7f24ac0ad380 2024-04-23T18:34:40.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.299+0000 7f24c8038700 1 -- 172.21.15.132:0/2539826066 --> [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f24c00635c0 con 0x7f24ac0ad380 2024-04-23T18:34:40.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.300+0000 7f24b6ffd700 1 -- 172.21.15.132:0/2539826066 <== osd.7 v2:172.21.15.148:6816/199627413 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f24c00635c0 con 0x7f24ac0ad380 2024-04-23T18:34:40.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.300+0000 7f24c8038700 1 -- 172.21.15.132:0/2539826066 >> [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413] conn(0x7f24ac0ad380 msgr2=0x7f24ac0af7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:40.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.300+0000 7f24c8038700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413] conn(0x7f24ac0ad380 0x7f24ac0af7a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:40.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.300+0000 7f24c8038700 1 -- 172.21.15.132:0/2539826066 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24ac0a3cd0 msgr2=0x7f24ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:40.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.300+0000 7f24c8038700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24ac0a3cd0 0x7f24ac0a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f24bc006fd0 tx=0x7f24bc008040 comp rx=0 tx=0).stop 2024-04-23T18:34:40.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.300+0000 7f24c8038700 1 -- 172.21.15.132:0/2539826066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c0101cf0 msgr2=0x7f24c019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:40.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.300+0000 7f24c8038700 1 --2- 172.21.15.132:0/2539826066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c0101cf0 0x7f24c019a580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009f50 tx=0x7f24b00037c0 comp rx=0 tx=0).stop 2024-04-23T18:34:40.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.300+0000 7f24c65d5700 1 -- 172.21.15.132:0/2539826066 reap_dead start 2024-04-23T18:34:40.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.300+0000 7f24c8038700 1 -- 172.21.15.132:0/2539826066 shutdown_connections 2024-04-23T18:34:40.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.300+0000 7f24c8038700 1 -- 172.21.15.132:0/2539826066 >> 172.21.15.132:0/2539826066 conn(0x7f24c0077f40 msgr2=0x7f24c0104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:40.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.300+0000 7f24c8038700 1 -- 172.21.15.132:0/2539826066 shutdown_connections 2024-04-23T18:34:40.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.300+0000 7f24c8038700 1 -- 172.21.15.132:0/2539826066 wait complete. 2024-04-23T18:34:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.178+0000 7f120590c700 1 -- 172.21.15.132:0/1451830277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f120006b1b0 msgr2=0x7f120006b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.178+0000 7f120590c700 1 --2- 172.21.15.132:0/1451830277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f120006b1b0 0x7f120006b630 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f11f40099c0 tx=0x7f11f4009cd0 comp rx=0 tx=0).stop 2024-04-23T18:34:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.178+0000 7f120590c700 1 -- 172.21.15.132:0/1451830277 shutdown_connections 2024-04-23T18:34:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.178+0000 7f120590c700 1 --2- 172.21.15.132:0/1451830277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f120006bb70 0x7f120010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.178+0000 7f120590c700 1 --2- 172.21.15.132:0/1451830277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1200108a50 0x7f1200108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.178+0000 7f120590c700 1 --2- 172.21.15.132:0/1451830277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f120006b1b0 0x7f120006b630 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.179+0000 7f120590c700 1 -- 172.21.15.132:0/1451830277 >> 172.21.15.132:0/1451830277 conn(0x7f1200077990 msgr2=0x7f1200077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.179+0000 7f120590c700 1 -- 172.21.15.132:0/1451830277 shutdown_connections 2024-04-23T18:34:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.179+0000 7f120590c700 1 -- 172.21.15.132:0/1451830277 wait complete. 2024-04-23T18:34:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.179+0000 7f120590c700 1 Processor -- start 2024-04-23T18:34:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f120590c700 1 -- start start 2024-04-23T18:34:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f120590c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f120006b1b0 0x7f1200199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f120590c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f120006bb70 0x7f1200199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f120590c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1200108a50 0x7f120019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f120590c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1200112c90 con 0x7f120006b1b0 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f120590c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1200112b10 con 0x7f1200108a50 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f120590c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1200112e10 con 0x7f120006bb70 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f11feffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f120006b1b0 0x7f1200199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f11feffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f120006b1b0 0x7f1200199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52986/0 (socket says 172.21.15.132:52986) 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f11feffd700 1 -- 172.21.15.132:0/4157210698 learned_addr learned my addr 172.21.15.132:0/4157210698 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f11ff7fe700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1200108a50 0x7f120019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f11feffd700 1 -- 172.21.15.132:0/4157210698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f120006bb70 msgr2=0x7f1200199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f11fe7fc700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f120006bb70 0x7f1200199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f11feffd700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f120006bb70 0x7f1200199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:41.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f11feffd700 1 -- 172.21.15.132:0/4157210698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1200108a50 msgr2=0x7f120019dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f11feffd700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1200108a50 0x7f120019dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f11feffd700 1 -- 172.21.15.132:0/4157210698 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120019e210 con 0x7f120006b1b0 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f11ff7fe700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1200108a50 0x7f120019dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.180+0000 7f11feffd700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f120006b1b0 0x7f1200199200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f11f000d950 tx=0x7f11f000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.181+0000 7f120490a700 1 -- 172.21.15.132:0/4157210698 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0016070 con 0x7f120006b1b0 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.181+0000 7f120590c700 1 -- 172.21.15.132:0/4157210698 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120019e500 con 0x7f120006b1b0 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.181+0000 7f120490a700 1 -- 172.21.15.132:0/4157210698 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f0010460 con 0x7f120006b1b0 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.181+0000 7f120490a700 1 -- 172.21.15.132:0/4157210698 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f000f620 con 0x7f120006b1b0 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.181+0000 7f120590c700 1 -- 172.21.15.132:0/4157210698 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12001a5d80 con 0x7f120006b1b0 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.181+0000 7f120590c700 1 -- 172.21.15.132:0/4157210698 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1200063150 con 0x7f120006b1b0 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.183+0000 7f120490a700 1 -- 172.21.15.132:0/4157210698 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f11f0019070 con 0x7f120006b1b0 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.183+0000 7f120490a700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f11e80a3cd0 0x7f11e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.184+0000 7f11fe7fc700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f11e80a3cd0 0x7f11e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.184+0000 7f120490a700 1 -- 172.21.15.132:0/4157210698 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f11f00d0620 con 0x7f120006b1b0 2024-04-23T18:34:41.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.184+0000 7f120490a700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653] conn(0x7f11e80ad4b0 0x7f11e80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:41.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.184+0000 7f120490a700 1 -- 172.21.15.132:0/4157210698 --> [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11e80aff80 con 0x7f11e80ad4b0 2024-04-23T18:34:41.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.184+0000 7f120490a700 1 -- 172.21.15.132:0/4157210698 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f11f00d0b00 con 0x7f120006b1b0 2024-04-23T18:34:41.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.184+0000 7f11fe7fc700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f11e80a3cd0 0x7f11e80a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f11f4003800 tx=0x7f11f4009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:41.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.184+0000 7f11ff7fe700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653] conn(0x7f11e80ad4b0 0x7f11e80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:41.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.185+0000 7f11ff7fe700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653] conn(0x7f11e80ad4b0 0x7f11e80af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:41.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.185+0000 7f120490a700 1 -- 172.21.15.132:0/4157210698 <== osd.5 v2:172.21.15.157:6808/3700977653 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11e80aff80 con 0x7f11e80ad4b0 2024-04-23T18:34:41.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.202+0000 7f120590c700 1 -- 172.21.15.132:0/4157210698 --> [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1200063580 con 0x7f11e80ad4b0 2024-04-23T18:34:41.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.203+0000 7f120490a700 1 -- 172.21.15.132:0/4157210698 <== osd.5 v2:172.21.15.157:6808/3700977653 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1200063580 con 0x7f11e80ad4b0 2024-04-23T18:34:41.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.203+0000 7f120590c700 1 -- 172.21.15.132:0/4157210698 >> [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653] conn(0x7f11e80ad4b0 msgr2=0x7f11e80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:41.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.203+0000 7f120590c700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653] conn(0x7f11e80ad4b0 0x7f11e80af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:41.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.203+0000 7f120590c700 1 -- 172.21.15.132:0/4157210698 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f11e80a3cd0 msgr2=0x7f11e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:41.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.203+0000 7f120590c700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f11e80a3cd0 0x7f11e80a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f11f4003800 tx=0x7f11f4009f10 comp rx=0 tx=0).stop 2024-04-23T18:34:41.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.203+0000 7f120590c700 1 -- 172.21.15.132:0/4157210698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f120006b1b0 msgr2=0x7f1200199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:41.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.203+0000 7f120590c700 1 --2- 172.21.15.132:0/4157210698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f120006b1b0 0x7f1200199200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f11f000d950 tx=0x7f11f000dd10 comp rx=0 tx=0).stop 2024-04-23T18:34:41.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.203+0000 7f11ff7fe700 1 -- 172.21.15.132:0/4157210698 reap_dead start 2024-04-23T18:34:41.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.203+0000 7f120590c700 1 -- 172.21.15.132:0/4157210698 shutdown_connections 2024-04-23T18:34:41.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.203+0000 7f120590c700 1 -- 172.21.15.132:0/4157210698 >> 172.21.15.132:0/4157210698 conn(0x7f1200077990 msgr2=0x7f12001007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:41.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.203+0000 7f120590c700 1 -- 172.21.15.132:0/4157210698 shutdown_connections 2024-04-23T18:34:41.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:40.203+0000 7f120590c700 1 -- 172.21.15.132:0/4157210698 wait complete. 2024-04-23T18:34:41.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:41 smithi148 ceph-mon[32311]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:34:41.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:41 smithi157 ceph-mon[32580]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:34:41.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:41 smithi132 ceph-mon[29178]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:34:41.856 INFO:teuthology.orchestra.run.smithi132.stdout:163208757299 2024-04-23T18:34:41.857 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-23T18:34:41.863 INFO:teuthology.orchestra.run.smithi132.stdout:107374182461 2024-04-23T18:34:41.863 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-23T18:34:42.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.602+0000 7f08bb6d6700 1 -- 172.21.15.132:0/2357202591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08b4101940 msgr2=0x7f08b4101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.602+0000 7f08bb6d6700 1 --2- 172.21.15.132:0/2357202591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08b4101940 0x7f08b4101dc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009990 tx=0x7f08a4009ca0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.602+0000 7f08bb6d6700 1 -- 172.21.15.132:0/2357202591 shutdown_connections 2024-04-23T18:34:42.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.602+0000 7f08bb6d6700 1 --2- 172.21.15.132:0/2357202591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b4102300 0x7f08b410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.602+0000 7f08bb6d6700 1 --2- 172.21.15.132:0/2357202591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08b4107990 0x7f08b4107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.602+0000 7f08bb6d6700 1 --2- 172.21.15.132:0/2357202591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08b4101940 0x7f08b4101dc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.602+0000 7f08bb6d6700 1 -- 172.21.15.132:0/2357202591 >> 172.21.15.132:0/2357202591 conn(0x7f08b40788f0 msgr2=0x7f08b4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08bb6d6700 1 -- 172.21.15.132:0/2357202591 shutdown_connections 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08bb6d6700 1 -- 172.21.15.132:0/2357202591 wait complete. 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08bb6d6700 1 Processor -- start 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08bb6d6700 1 -- start start 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08bb6d6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08b4101940 0x7f08b4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08bb6d6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b4102300 0x7f08b4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08bb6d6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08b4107990 0x7f08b419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08bb6d6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f08b4111ae0 con 0x7f08b4101940 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08bb6d6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f08b4111960 con 0x7f08b4107990 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08bb6d6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08b4111c60 con 0x7f08b4102300 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08b9472700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08b4101940 0x7f08b4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08b9472700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08b4101940 0x7f08b4197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53032/0 (socket says 172.21.15.132:53032) 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.603+0000 7f08b9472700 1 -- 172.21.15.132:0/496657142 learned_addr learned my addr 172.21.15.132:0/496657142 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08b8c71700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b4102300 0x7f08b4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08b9c73700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08b4107990 0x7f08b419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08b9472700 1 -- 172.21.15.132:0/496657142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b4102300 msgr2=0x7f08b4198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08b9472700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b4102300 0x7f08b4198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08b9472700 1 -- 172.21.15.132:0/496657142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08b4107990 msgr2=0x7f08b419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08b9472700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08b4107990 0x7f08b419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08b9472700 1 -- 172.21.15.132:0/496657142 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b419d000 con 0x7f08b4101940 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08b8c71700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b4102300 0x7f08b4198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08b9c73700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08b4107990 0x7f08b419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08b9472700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08b4101940 0x7f08b4197ff0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f08b000c9b0 tx=0x7f08b000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08aa7fc700 1 -- 172.21.15.132:0/496657142 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0007b00 con 0x7f08b4101940 2024-04-23T18:34:42.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08aa7fc700 1 -- 172.21.15.132:0/496657142 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b0007ca0 con 0x7f08b4101940 2024-04-23T18:34:42.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08bb6d6700 1 -- 172.21.15.132:0/496657142 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b419d2f0 con 0x7f08b4101940 2024-04-23T18:34:42.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08aa7fc700 1 -- 172.21.15.132:0/496657142 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0018730 con 0x7f08b4101940 2024-04-23T18:34:42.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.604+0000 7f08bb6d6700 1 -- 172.21.15.132:0/496657142 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b41a4b70 con 0x7f08b4101940 2024-04-23T18:34:42.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.605+0000 7f08bb6d6700 1 -- 172.21.15.132:0/496657142 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08b4063110 con 0x7f08b4101940 2024-04-23T18:34:42.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.607+0000 7f08aa7fc700 1 -- 172.21.15.132:0/496657142 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f08b000f450 con 0x7f08b4101940 2024-04-23T18:34:42.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.607+0000 7f08aa7fc700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f08a00a3cd0 0x7f08a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.608+0000 7f08b8c71700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f08a00a3cd0 0x7f08a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.608+0000 7f08aa7fc700 1 -- 172.21.15.132:0/496657142 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f08b00d0860 con 0x7f08b4101940 2024-04-23T18:34:42.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.608+0000 7f08aa7fc700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299] conn(0x7f08a00ad4b0 0x7f08a00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.608+0000 7f08aa7fc700 1 -- 172.21.15.132:0/496657142 --> [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08b0004500 con 0x7f08a00ad4b0 2024-04-23T18:34:42.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.608+0000 7f08aa7fc700 1 -- 172.21.15.132:0/496657142 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f08b00d0d40 con 0x7f08b4101940 2024-04-23T18:34:42.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.608+0000 7f08b8c71700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f08a00a3cd0 0x7f08a00a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f08a400b5c0 tx=0x7f08a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:42.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.608+0000 7f08b9c73700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299] conn(0x7f08a00ad4b0 0x7f08a00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.609+0000 7f08b9c73700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299] conn(0x7f08a00ad4b0 0x7f08a00af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:42.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.610+0000 7f08aa7fc700 1 -- 172.21.15.132:0/496657142 <== osd.10 v2:172.21.15.148:6824/2494142299 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08b0004500 con 0x7f08a00ad4b0 2024-04-23T18:34:42.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.627+0000 7f08bb6d6700 1 -- 172.21.15.132:0/496657142 --> [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f08b4063590 con 0x7f08a00ad4b0 2024-04-23T18:34:42.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.627+0000 7f08aa7fc700 1 -- 172.21.15.132:0/496657142 <== osd.10 v2:172.21.15.148:6824/2494142299 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f08b4063590 con 0x7f08a00ad4b0 2024-04-23T18:34:42.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.628+0000 7f08bb6d6700 1 -- 172.21.15.132:0/496657142 >> [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299] conn(0x7f08a00ad4b0 msgr2=0x7f08a00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.628+0000 7f08bb6d6700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299] conn(0x7f08a00ad4b0 0x7f08a00af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.628+0000 7f08bb6d6700 1 -- 172.21.15.132:0/496657142 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f08a00a3cd0 msgr2=0x7f08a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.628+0000 7f08bb6d6700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f08a00a3cd0 0x7f08a00a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f08a400b5c0 tx=0x7f08a4005fb0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.628+0000 7f08bb6d6700 1 -- 172.21.15.132:0/496657142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08b4101940 msgr2=0x7f08b4197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.628+0000 7f08bb6d6700 1 --2- 172.21.15.132:0/496657142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08b4101940 0x7f08b4197ff0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f08b000c9b0 tx=0x7f08b000cd70 comp rx=0 tx=0).stop 2024-04-23T18:34:42.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.628+0000 7f08b9c73700 1 -- 172.21.15.132:0/496657142 reap_dead start 2024-04-23T18:34:42.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.628+0000 7f08bb6d6700 1 -- 172.21.15.132:0/496657142 shutdown_connections 2024-04-23T18:34:42.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.628+0000 7f08bb6d6700 1 -- 172.21.15.132:0/496657142 >> 172.21.15.132:0/496657142 conn(0x7f08b40788f0 msgr2=0x7f08b40ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:42.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.628+0000 7f08bb6d6700 1 -- 172.21.15.132:0/496657142 shutdown_connections 2024-04-23T18:34:42.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.628+0000 7f08bb6d6700 1 -- 172.21.15.132:0/496657142 wait complete. 2024-04-23T18:34:42.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.498+0000 7f981d87d700 1 -- 172.21.15.132:0/2397562334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9818107970 msgr2=0x7f981806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.498+0000 7f981d87d700 1 --2- 172.21.15.132:0/2397562334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9818107970 0x7f981806a750 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9808009990 tx=0x7f9808009ca0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.498+0000 7f981d87d700 1 -- 172.21.15.132:0/2397562334 shutdown_connections 2024-04-23T18:34:42.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.498+0000 7f981d87d700 1 --2- 172.21.15.132:0/2397562334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9818105480 0x7f9818105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.498+0000 7f981d87d700 1 --2- 172.21.15.132:0/2397562334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9818107970 0x7f981806a750 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.498+0000 7f981d87d700 1 --2- 172.21.15.132:0/2397562334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f981806ac90 0x7f981806b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.498+0000 7f981d87d700 1 -- 172.21.15.132:0/2397562334 >> 172.21.15.132:0/2397562334 conn(0x7f98180788f0 msgr2=0x7f9818078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:42.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.498+0000 7f981d87d700 1 -- 172.21.15.132:0/2397562334 shutdown_connections 2024-04-23T18:34:42.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.498+0000 7f981d87d700 1 -- 172.21.15.132:0/2397562334 wait complete. 2024-04-23T18:34:42.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.499+0000 7f981d87d700 1 Processor -- start 2024-04-23T18:34:42.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.499+0000 7f981d87d700 1 -- start start 2024-04-23T18:34:42.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f981d87d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f981806ac90 0x7f981819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f981d87d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9818105480 0x7f981819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f981d87d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9818107970 0x7f98181a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f981d87d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f98181191d0 con 0x7f981806ac90 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f981d87d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9818119050 con 0x7f9818107970 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f981d87d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9818119350 con 0x7f9818105480 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f981806ac90 0x7f981819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f981806ac90 0x7f981819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53016/0 (socket says 172.21.15.132:53016) 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f9816ffd700 1 -- 172.21.15.132:0/317115388 learned_addr learned my addr 172.21.15.132:0/317115388 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f98167fc700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9818105480 0x7f981819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f9816ffd700 1 -- 172.21.15.132:0/317115388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9818105480 msgr2=0x7f981819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f9816ffd700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9818105480 0x7f981819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f9816ffd700 1 -- 172.21.15.132:0/317115388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9818107970 msgr2=0x7f98181a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f9816ffd700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9818107970 0x7f98181a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f9816ffd700 1 -- 172.21.15.132:0/317115388 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98181a4750 con 0x7f981806ac90 2024-04-23T18:34:42.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.500+0000 7f9816ffd700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f981806ac90 0x7f981819f740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9808000c00 tx=0x7f98080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.501+0000 7f981c87b700 1 -- 172.21.15.132:0/317115388 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808027070 con 0x7f981806ac90 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.501+0000 7f981c87b700 1 -- 172.21.15.132:0/317115388 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9808003f20 con 0x7f981806ac90 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.501+0000 7f981d87d700 1 -- 172.21.15.132:0/317115388 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98181a49e0 con 0x7f981806ac90 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.501+0000 7f981c87b700 1 -- 172.21.15.132:0/317115388 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808022cf0 con 0x7f981806ac90 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.501+0000 7f981d87d700 1 -- 172.21.15.132:0/317115388 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98181ac2c0 con 0x7f981806ac90 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.501+0000 7f981d87d700 1 -- 172.21.15.132:0/317115388 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9818101310 con 0x7f981806ac90 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.503+0000 7f981c87b700 1 -- 172.21.15.132:0/317115388 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f980802a070 con 0x7f981806ac90 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.503+0000 7f981c87b700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98000a3cd0 0x7f98000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.503+0000 7f981c87b700 1 -- 172.21.15.132:0/317115388 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f98080d17f0 con 0x7f981806ac90 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.503+0000 7f98167fc700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98000a3cd0 0x7f98000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.503+0000 7f981c87b700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226] conn(0x7f98000ad4b0 0x7f98000af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.503+0000 7f981c87b700 1 -- 172.21.15.132:0/317115388 --> [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98080095d0 con 0x7f98000ad4b0 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.503+0000 7f981c87b700 1 -- 172.21.15.132:0/317115388 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f98080d1cd0 con 0x7f981806ac90 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.504+0000 7f98177fe700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226] conn(0x7f98000ad4b0 0x7f98000af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.504+0000 7f98167fc700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98000a3cd0 0x7f98000a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9810009550 tx=0x7f98100091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:42.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.504+0000 7f98177fe700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226] conn(0x7f98000ad4b0 0x7f98000af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:42.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.505+0000 7f981c87b700 1 -- 172.21.15.132:0/317115388 <== osd.6 v2:172.21.15.132:6818/3702092226 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98080095d0 con 0x7f98000ad4b0 2024-04-23T18:34:42.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.521+0000 7f981d87d700 1 -- 172.21.15.132:0/317115388 --> [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9818101f60 con 0x7f98000ad4b0 2024-04-23T18:34:42.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.521+0000 7f981c87b700 1 -- 172.21.15.132:0/317115388 <== osd.6 v2:172.21.15.132:6818/3702092226 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9818101f60 con 0x7f98000ad4b0 2024-04-23T18:34:42.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.521+0000 7f981d87d700 1 -- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226] conn(0x7f98000ad4b0 msgr2=0x7f98000af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.521+0000 7f981d87d700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226] conn(0x7f98000ad4b0 0x7f98000af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.522+0000 7f981d87d700 1 -- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98000a3cd0 msgr2=0x7f98000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.522+0000 7f981d87d700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98000a3cd0 0x7f98000a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9810009550 tx=0x7f98100091b0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.522+0000 7f981d87d700 1 -- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f981806ac90 msgr2=0x7f981819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.522+0000 7f981d87d700 1 --2- 172.21.15.132:0/317115388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f981806ac90 0x7f981819f740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9808000c00 tx=0x7f98080047b0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.522+0000 7f98177fe700 1 -- 172.21.15.132:0/317115388 reap_dead start 2024-04-23T18:34:42.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.522+0000 7f981d87d700 1 -- 172.21.15.132:0/317115388 shutdown_connections 2024-04-23T18:34:42.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.522+0000 7f981d87d700 1 -- 172.21.15.132:0/317115388 >> 172.21.15.132:0/317115388 conn(0x7f98180788f0 msgr2=0x7f9818100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:42.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.522+0000 7f981d87d700 1 -- 172.21.15.132:0/317115388 shutdown_connections 2024-04-23T18:34:42.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:42.522+0000 7f981d87d700 1 -- 172.21.15.132:0/317115388 wait complete. 2024-04-23T18:34:43.464 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:43 smithi132 ceph-mon[29178]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:34:43.465 INFO:teuthology.orchestra.run.smithi132.stdout:197568495664 2024-04-23T18:34:43.465 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-23T18:34:43.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:43 smithi148 ceph-mon[32311]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:34:43.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:43 smithi157 ceph-mon[32580]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:34:43.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.660+0000 7f45db27a700 1 -- 172.21.15.132:0/754601577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d4108a30 msgr2=0x7f45d4108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.660+0000 7f45db27a700 1 --2- 172.21.15.132:0/754601577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d4108a30 0x7f45d4108e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009990 tx=0x7f45d0009ca0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.660+0000 7f45db27a700 1 -- 172.21.15.132:0/754601577 shutdown_connections 2024-04-23T18:34:43.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.660+0000 7f45db27a700 1 --2- 172.21.15.132:0/754601577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d406b0f0 0x7f45d410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.660+0000 7f45db27a700 1 --2- 172.21.15.132:0/754601577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d4108a30 0x7f45d4108e10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.660+0000 7f45db27a700 1 --2- 172.21.15.132:0/754601577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d406a750 0x7f45d406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.660+0000 7f45db27a700 1 -- 172.21.15.132:0/754601577 >> 172.21.15.132:0/754601577 conn(0x7f45d40788f0 msgr2=0x7f45d4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:43.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.660+0000 7f45db27a700 1 -- 172.21.15.132:0/754601577 shutdown_connections 2024-04-23T18:34:43.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.660+0000 7f45db27a700 1 -- 172.21.15.132:0/754601577 wait complete. 2024-04-23T18:34:43.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.661+0000 7f45db27a700 1 Processor -- start 2024-04-23T18:34:43.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.661+0000 7f45db27a700 1 -- start start 2024-04-23T18:34:43.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.661+0000 7f45db27a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d406a750 0x7f45d4199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.661+0000 7f45db27a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d406b0f0 0x7f45d4199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45d9016700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d406a750 0x7f45d4199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45db27a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4108a30 0x7f45d419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45db27a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4112c30 con 0x7f45d406a750 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45db27a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4112ab0 con 0x7f45d406b0f0 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45db27a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4112db0 con 0x7f45d4108a30 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45d9016700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d406a750 0x7f45d4199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53046/0 (socket says 172.21.15.132:53046) 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45d9016700 1 -- 172.21.15.132:0/3020020499 learned_addr learned my addr 172.21.15.132:0/3020020499 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45d9016700 1 -- 172.21.15.132:0/3020020499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4108a30 msgr2=0x7f45d419d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45d9016700 1 --2- 172.21.15.132:0/3020020499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4108a30 0x7f45d419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45d9016700 1 -- 172.21.15.132:0/3020020499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d406b0f0 msgr2=0x7f45d4199680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45d9016700 1 --2- 172.21.15.132:0/3020020499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d406b0f0 0x7f45d4199680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45d9016700 1 -- 172.21.15.132:0/3020020499 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d419e100 con 0x7f45d406a750 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45d9016700 1 --2- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d406a750 0x7f45d4199140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f45d0000c00 tx=0x7f45d000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.662+0000 7f45ca7fc700 1 -- 172.21.15.132:0/3020020499 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d002c070 con 0x7f45d406a750 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.663+0000 7f45ca7fc700 1 -- 172.21.15.132:0/3020020499 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45d001ce00 con 0x7f45d406a750 2024-04-23T18:34:43.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.663+0000 7f45ca7fc700 1 -- 172.21.15.132:0/3020020499 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d000f6c0 con 0x7f45d406a750 2024-04-23T18:34:43.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.663+0000 7f45db27a700 1 -- 172.21.15.132:0/3020020499 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d419e390 con 0x7f45d406a750 2024-04-23T18:34:43.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.663+0000 7f45db27a700 1 -- 172.21.15.132:0/3020020499 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d4101940 con 0x7f45d406a750 2024-04-23T18:34:43.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.664+0000 7f45db27a700 1 -- 172.21.15.132:0/3020020499 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45d419e820 con 0x7f45d406a750 2024-04-23T18:34:43.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.666+0000 7f45ca7fc700 1 -- 172.21.15.132:0/3020020499 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f45d002f070 con 0x7f45d406a750 2024-04-23T18:34:43.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.666+0000 7f45ca7fc700 1 --2- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45c00a3d50 0x7f45c00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.667+0000 7f45d8815700 1 --2- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45c00a3d50 0x7f45c00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.667+0000 7f45ca7fc700 1 -- 172.21.15.132:0/3020020499 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f45d00d5de0 con 0x7f45d406a750 2024-04-23T18:34:43.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.667+0000 7f45ca7fc700 1 --2- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150] conn(0x7f45c00ad530 0x7f45c00af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:43.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.667+0000 7f45ca7fc700 1 -- 172.21.15.132:0/3020020499 --> [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45c00b0000 con 0x7f45c00ad530 2024-04-23T18:34:43.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.667+0000 7f45ca7fc700 1 -- 172.21.15.132:0/3020020499 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f45d00d62c0 con 0x7f45d406a750 2024-04-23T18:34:43.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.667+0000 7f45d9817700 1 --2- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150] conn(0x7f45c00ad530 0x7f45c00af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:43.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.667+0000 7f45d8815700 1 --2- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45c00a3d50 0x7f45c00a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f45d4078fd0 tx=0x7f45c4005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:43.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.668+0000 7f45d9817700 1 --2- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150] conn(0x7f45c00ad530 0x7f45c00af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:43.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.668+0000 7f45ca7fc700 1 -- 172.21.15.132:0/3020020499 <== osd.0 v2:172.21.15.132:6802/1754141150 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45c00b0000 con 0x7f45c00ad530 2024-04-23T18:34:43.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.685+0000 7f45db27a700 1 -- 172.21.15.132:0/3020020499 --> [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f45d4063420 con 0x7f45c00ad530 2024-04-23T18:34:43.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.686+0000 7f45ca7fc700 1 -- 172.21.15.132:0/3020020499 <== osd.0 v2:172.21.15.132:6802/1754141150 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f45d4063420 con 0x7f45c00ad530 2024-04-23T18:34:43.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.686+0000 7f45db27a700 1 -- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150] conn(0x7f45c00ad530 msgr2=0x7f45c00af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.686+0000 7f45db27a700 1 --2- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150] conn(0x7f45c00ad530 0x7f45c00af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.686+0000 7f45db27a700 1 -- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45c00a3d50 msgr2=0x7f45c00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.686+0000 7f45db27a700 1 --2- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45c00a3d50 0x7f45c00a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f45d4078fd0 tx=0x7f45c4005c90 comp rx=0 tx=0).stop 2024-04-23T18:34:43.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.686+0000 7f45db27a700 1 -- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d406a750 msgr2=0x7f45d4199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:43.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.686+0000 7f45db27a700 1 --2- 172.21.15.132:0/3020020499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45d406a750 0x7f45d4199140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f45d0000c00 tx=0x7f45d000bab0 comp rx=0 tx=0).stop 2024-04-23T18:34:43.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.686+0000 7f45d9817700 1 -- 172.21.15.132:0/3020020499 reap_dead start 2024-04-23T18:34:43.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.686+0000 7f45db27a700 1 -- 172.21.15.132:0/3020020499 shutdown_connections 2024-04-23T18:34:43.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.686+0000 7f45db27a700 1 -- 172.21.15.132:0/3020020499 >> 172.21.15.132:0/3020020499 conn(0x7f45d40788f0 msgr2=0x7f45d40ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:43.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.687+0000 7f45db27a700 1 -- 172.21.15.132:0/3020020499 shutdown_connections 2024-04-23T18:34:43.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:43.687+0000 7f45db27a700 1 -- 172.21.15.132:0/3020020499 wait complete. 2024-04-23T18:34:44.617 INFO:teuthology.orchestra.run.smithi132.stdout:292057776159 2024-04-23T18:34:44.618 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-23T18:34:45.243 INFO:teuthology.orchestra.run.smithi132.stdout:313532612635 2024-04-23T18:34:45.243 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-23T18:34:45.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:45 smithi148 ceph-mon[32311]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-04-23T18:34:45.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:45 smithi157 ceph-mon[32580]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-04-23T18:34:45.545 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:45 smithi132 ceph-mon[29178]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-04-23T18:34:45.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.560+0000 7fab7a150700 1 -- 172.21.15.132:0/1860165640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab741022f0 msgr2=0x7fab7410e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:45.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.560+0000 7fab7a150700 1 --2- 172.21.15.132:0/1860165640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab741022f0 0x7fab7410e7e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00b0a0 tx=0x7fab6c00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:34:45.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.560+0000 7fab7a150700 1 -- 172.21.15.132:0/1860165640 shutdown_connections 2024-04-23T18:34:45.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.560+0000 7fab7a150700 1 --2- 172.21.15.132:0/1860165640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab741022f0 0x7fab7410e7e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:45.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.560+0000 7fab7a150700 1 --2- 172.21.15.132:0/1860165640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab74107980 0x7fab74107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:45.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.560+0000 7fab7a150700 1 --2- 172.21.15.132:0/1860165640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab74101930 0x7fab74101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:45.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.560+0000 7fab7a150700 1 -- 172.21.15.132:0/1860165640 >> 172.21.15.132:0/1860165640 conn(0x7fab740788b0 msgr2=0x7fab74078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:45.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.560+0000 7fab7a150700 1 -- 172.21.15.132:0/1860165640 shutdown_connections 2024-04-23T18:34:45.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.560+0000 7fab7a150700 1 -- 172.21.15.132:0/1860165640 wait complete. 2024-04-23T18:34:45.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.560+0000 7fab7a150700 1 Processor -- start 2024-04-23T18:34:45.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab7a150700 1 -- start start 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab7a150700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab74101930 0x7fab74199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab7a150700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab741022f0 0x7fab741995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab7a150700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab74107980 0x7fab7419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab7a150700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fab74111ae0 con 0x7fab74101930 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab7a150700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab74111960 con 0x7fab741022f0 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab7a150700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab74111c60 con 0x7fab74107980 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab737fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab74101930 0x7fab74199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab737fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab74101930 0x7fab74199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53062/0 (socket says 172.21.15.132:53062) 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab737fe700 1 -- 172.21.15.132:0/4231194719 learned_addr learned my addr 172.21.15.132:0/4231194719 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab72ffd700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab741022f0 0x7fab741995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab73fff700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab74107980 0x7fab7419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab737fe700 1 -- 172.21.15.132:0/4231194719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab74107980 msgr2=0x7fab7419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:45.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab737fe700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab74107980 0x7fab7419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:45.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab737fe700 1 -- 172.21.15.132:0/4231194719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab741022f0 msgr2=0x7fab741995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:45.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab737fe700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab741022f0 0x7fab741995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:45.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.561+0000 7fab737fe700 1 -- 172.21.15.132:0/4231194719 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7419e080 con 0x7fab74101930 2024-04-23T18:34:45.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.562+0000 7fab73fff700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab74107980 0x7fab7419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:45.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.562+0000 7fab737fe700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab74101930 0x7fab74199070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fab6400d950 tx=0x7fab6400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:45.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.562+0000 7fab70ff9700 1 -- 172.21.15.132:0/4231194719 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64016070 con 0x7fab74101930 2024-04-23T18:34:45.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.562+0000 7fab7a150700 1 -- 172.21.15.132:0/4231194719 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7419e370 con 0x7fab74101930 2024-04-23T18:34:45.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.562+0000 7fab70ff9700 1 -- 172.21.15.132:0/4231194719 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab64009df0 con 0x7fab74101930 2024-04-23T18:34:45.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.562+0000 7fab7a150700 1 -- 172.21.15.132:0/4231194719 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab741a5bf0 con 0x7fab74101930 2024-04-23T18:34:45.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.563+0000 7fab70ff9700 1 -- 172.21.15.132:0/4231194719 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6400f3c0 con 0x7fab74101930 2024-04-23T18:34:45.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.563+0000 7fab7a150700 1 -- 172.21.15.132:0/4231194719 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab740631d0 con 0x7fab74101930 2024-04-23T18:34:45.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.565+0000 7fab70ff9700 1 -- 172.21.15.132:0/4231194719 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fab64019070 con 0x7fab74101930 2024-04-23T18:34:45.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.565+0000 7fab70ff9700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fab5c0a3cd0 0x7fab5c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:45.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.566+0000 7fab72ffd700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fab5c0a3cd0 0x7fab5c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:45.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.566+0000 7fab70ff9700 1 -- 172.21.15.132:0/4231194719 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fab640d0400 con 0x7fab74101930 2024-04-23T18:34:45.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.566+0000 7fab70ff9700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159] conn(0x7fab5c0ad4b0 0x7fab5c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:45.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.566+0000 7fab70ff9700 1 -- 172.21.15.132:0/4231194719 --> [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab5c0aff80 con 0x7fab5c0ad4b0 2024-04-23T18:34:45.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.566+0000 7fab70ff9700 1 -- 172.21.15.132:0/4231194719 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fab640d08e0 con 0x7fab74101930 2024-04-23T18:34:45.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.566+0000 7fab73fff700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159] conn(0x7fab5c0ad4b0 0x7fab5c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:45.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.567+0000 7fab72ffd700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fab5c0a3cd0 0x7fab5c0a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fab74078f60 tx=0x7fab6800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:45.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.567+0000 7fab73fff700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159] conn(0x7fab5c0ad4b0 0x7fab5c0af8d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:45.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.568+0000 7fab70ff9700 1 -- 172.21.15.132:0/4231194719 <== osd.8 v2:172.21.15.157:6816/3005625159 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab5c0aff80 con 0x7fab5c0ad4b0 2024-04-23T18:34:45.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.585+0000 7fab7a150700 1 -- 172.21.15.132:0/4231194719 --> [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fab74063600 con 0x7fab5c0ad4b0 2024-04-23T18:34:45.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.585+0000 7fab70ff9700 1 -- 172.21.15.132:0/4231194719 <== osd.8 v2:172.21.15.157:6816/3005625159 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fab74063600 con 0x7fab5c0ad4b0 2024-04-23T18:34:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.585+0000 7fab7a150700 1 -- 172.21.15.132:0/4231194719 >> [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159] conn(0x7fab5c0ad4b0 msgr2=0x7fab5c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.585+0000 7fab7a150700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159] conn(0x7fab5c0ad4b0 0x7fab5c0af8d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.586+0000 7fab7a150700 1 -- 172.21.15.132:0/4231194719 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fab5c0a3cd0 msgr2=0x7fab5c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.586+0000 7fab7a150700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fab5c0a3cd0 0x7fab5c0a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fab74078f60 tx=0x7fab6800b3c0 comp rx=0 tx=0).stop 2024-04-23T18:34:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.586+0000 7fab7a150700 1 -- 172.21.15.132:0/4231194719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab74101930 msgr2=0x7fab74199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.586+0000 7fab7a150700 1 --2- 172.21.15.132:0/4231194719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab74101930 0x7fab74199070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fab6400d950 tx=0x7fab6400dc60 comp rx=0 tx=0).stop 2024-04-23T18:34:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.586+0000 7fab73fff700 1 -- 172.21.15.132:0/4231194719 reap_dead start 2024-04-23T18:34:45.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.586+0000 7fab7a150700 1 -- 172.21.15.132:0/4231194719 shutdown_connections 2024-04-23T18:34:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.586+0000 7fab7a150700 1 -- 172.21.15.132:0/4231194719 >> 172.21.15.132:0/4231194719 conn(0x7fab740788b0 msgr2=0x7fab7406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.586+0000 7fab7a150700 1 -- 172.21.15.132:0/4231194719 shutdown_connections 2024-04-23T18:34:45.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:45.586+0000 7fab7a150700 1 -- 172.21.15.132:0/4231194719 wait complete. 2024-04-23T18:34:46.062 INFO:teuthology.orchestra.run.smithi132.stdout:34359738446 2024-04-23T18:34:46.063 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-23T18:34:46.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.272+0000 7fad4455a700 1 -- 172.21.15.132:0/321442368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad3c102ad0 msgr2=0x7fad3c102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.272+0000 7fad4455a700 1 --2- 172.21.15.132:0/321442368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad3c102ad0 0x7fad3c102f50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009990 tx=0x7fad2c009ca0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.273+0000 7fad4455a700 1 -- 172.21.15.132:0/321442368 shutdown_connections 2024-04-23T18:34:46.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.273+0000 7fad4455a700 1 --2- 172.21.15.132:0/321442368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad3c103490 0x7fad3c10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.273+0000 7fad4455a700 1 --2- 172.21.15.132:0/321442368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad3c108aa0 0x7fad3c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.273+0000 7fad4455a700 1 --2- 172.21.15.132:0/321442368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad3c102ad0 0x7fad3c102f50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.273+0000 7fad4455a700 1 -- 172.21.15.132:0/321442368 >> 172.21.15.132:0/321442368 conn(0x7fad3c079c30 msgr2=0x7fad3c07a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:46.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.273+0000 7fad4455a700 1 -- 172.21.15.132:0/321442368 shutdown_connections 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.273+0000 7fad4455a700 1 -- 172.21.15.132:0/321442368 wait complete. 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.273+0000 7fad4455a700 1 Processor -- start 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.273+0000 7fad4455a700 1 -- start start 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.273+0000 7fad4455a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad3c102ad0 0x7fad3c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.273+0000 7fad4455a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad3c103490 0x7fad3c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.273+0000 7fad4455a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad3c108aa0 0x7fad3c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad4455a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c112cd0 con 0x7fad3c108aa0 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad4455a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c112b50 con 0x7fad3c102ad0 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad4455a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c112e50 con 0x7fad3c103490 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad42af7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad3c108aa0 0x7fad3c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad42af7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad3c108aa0 0x7fad3c0757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53086/0 (socket says 172.21.15.132:53086) 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad42af7700 1 -- 172.21.15.132:0/1038783403 learned_addr learned my addr 172.21.15.132:0/1038783403 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad41af5700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad3c103490 0x7fad3c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad422f6700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad3c102ad0 0x7fad3c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad42af7700 1 -- 172.21.15.132:0/1038783403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad3c103490 msgr2=0x7fad3c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad42af7700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad3c103490 0x7fad3c075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad42af7700 1 -- 172.21.15.132:0/1038783403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad3c102ad0 msgr2=0x7fad3c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad42af7700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad3c102ad0 0x7fad3c074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad42af7700 1 -- 172.21.15.132:0/1038783403 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c1a76a0 con 0x7fad3c108aa0 2024-04-23T18:34:46.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad42af7700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad3c108aa0 0x7fad3c0757c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fad34007ee0 tx=0x7fad3400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:46.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad337fe700 1 -- 172.21.15.132:0/1038783403 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34004bb0 con 0x7fad3c108aa0 2024-04-23T18:34:46.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad337fe700 1 -- 172.21.15.132:0/1038783403 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad34004d50 con 0x7fad3c108aa0 2024-04-23T18:34:46.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad4455a700 1 -- 172.21.15.132:0/1038783403 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3c1a7990 con 0x7fad3c108aa0 2024-04-23T18:34:46.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.274+0000 7fad337fe700 1 -- 172.21.15.132:0/1038783403 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34005920 con 0x7fad3c108aa0 2024-04-23T18:34:46.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.275+0000 7fad4455a700 1 -- 172.21.15.132:0/1038783403 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3c1a7e80 con 0x7fad3c108aa0 2024-04-23T18:34:46.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.275+0000 7fad4455a700 1 -- 172.21.15.132:0/1038783403 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad3c0630d0 con 0x7fad3c108aa0 2024-04-23T18:34:46.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.277+0000 7fad337fe700 1 -- 172.21.15.132:0/1038783403 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fad3401a020 con 0x7fad3c108aa0 2024-04-23T18:34:46.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.278+0000 7fad337fe700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fad280a3cd0 0x7fad280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.278+0000 7fad422f6700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fad280a3cd0 0x7fad280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.278+0000 7fad337fe700 1 -- 172.21.15.132:0/1038783403 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fad340d05a0 con 0x7fad3c108aa0 2024-04-23T18:34:46.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.278+0000 7fad337fe700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818] conn(0x7fad280ad4b0 0x7fad280af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.278+0000 7fad337fe700 1 -- 172.21.15.132:0/1038783403 --> [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad340020b0 con 0x7fad280ad4b0 2024-04-23T18:34:46.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.278+0000 7fad337fe700 1 -- 172.21.15.132:0/1038783403 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fad340d0950 con 0x7fad3c108aa0 2024-04-23T18:34:46.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.278+0000 7fad422f6700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fad280a3cd0 0x7fad280a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fad3800abd0 tx=0x7fad3800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:46.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.278+0000 7fad41af5700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818] conn(0x7fad280ad4b0 0x7fad280af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.279+0000 7fad41af5700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818] conn(0x7fad280ad4b0 0x7fad280af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:46.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.280+0000 7fad337fe700 1 -- 172.21.15.132:0/1038783403 <== osd.11 v2:172.21.15.157:6824/2341045818 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fad340020b0 con 0x7fad280ad4b0 2024-04-23T18:34:46.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.297+0000 7fad4455a700 1 -- 172.21.15.132:0/1038783403 --> [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fad3c0634d0 con 0x7fad280ad4b0 2024-04-23T18:34:46.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.298+0000 7fad337fe700 1 -- 172.21.15.132:0/1038783403 <== osd.11 v2:172.21.15.157:6824/2341045818 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fad3c0634d0 con 0x7fad280ad4b0 2024-04-23T18:34:46.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.298+0000 7fad4455a700 1 -- 172.21.15.132:0/1038783403 >> [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818] conn(0x7fad280ad4b0 msgr2=0x7fad280af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.298+0000 7fad4455a700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818] conn(0x7fad280ad4b0 0x7fad280af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.298+0000 7fad4455a700 1 -- 172.21.15.132:0/1038783403 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fad280a3cd0 msgr2=0x7fad280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.298+0000 7fad4455a700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fad280a3cd0 0x7fad280a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fad3800abd0 tx=0x7fad3800a380 comp rx=0 tx=0).stop 2024-04-23T18:34:46.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.298+0000 7fad4455a700 1 -- 172.21.15.132:0/1038783403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad3c108aa0 msgr2=0x7fad3c0757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.298+0000 7fad4455a700 1 --2- 172.21.15.132:0/1038783403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad3c108aa0 0x7fad3c0757c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fad34007ee0 tx=0x7fad3400d3b0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.299+0000 7fad42af7700 1 -- 172.21.15.132:0/1038783403 reap_dead start 2024-04-23T18:34:46.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.299+0000 7fad4455a700 1 -- 172.21.15.132:0/1038783403 shutdown_connections 2024-04-23T18:34:46.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.299+0000 7fad4455a700 1 -- 172.21.15.132:0/1038783403 >> 172.21.15.132:0/1038783403 conn(0x7fad3c079c30 msgr2=0x7fad3c100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:46.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.299+0000 7fad4455a700 1 -- 172.21.15.132:0/1038783403 shutdown_connections 2024-04-23T18:34:46.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:46.299+0000 7fad4455a700 1 -- 172.21.15.132:0/1038783403 wait complete. 2024-04-23T18:34:46.917 INFO:teuthology.orchestra.run.smithi132.stdout:223338299434 2024-04-23T18:34:46.917 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-23T18:34:47.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:47 smithi148 ceph-mon[32311]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:34:47.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:47 smithi157 ceph-mon[32580]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:34:47.605 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:47 smithi132 ceph-mon[29178]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:34:47.946 INFO:teuthology.orchestra.run.smithi132.stdout:257698037798 2024-04-23T18:34:47.946 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-23T18:34:48.170 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:48.584 INFO:teuthology.orchestra.run.smithi132.stdout:347892350998 2024-04-23T18:34:48.585 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-23T18:34:48.863 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:48.864 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:48.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.872+0000 7f1f0d94b700 1 -- 172.21.15.132:0/262305197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f08102300 msgr2=0x7f1f0810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:48.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.872+0000 7f1f0d94b700 1 --2- 172.21.15.132:0/262305197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f08102300 0x7f1f0810e7f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000b0a0 tx=0x7f1f0000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.872+0000 7f1f0d94b700 1 -- 172.21.15.132:0/262305197 shutdown_connections 2024-04-23T18:34:48.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.872+0000 7f1f0d94b700 1 --2- 172.21.15.132:0/262305197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f08102300 0x7f1f0810e7f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.872+0000 7f1f0d94b700 1 --2- 172.21.15.132:0/262305197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f08107990 0x7f1f08107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.872+0000 7f1f0d94b700 1 --2- 172.21.15.132:0/262305197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f08101940 0x7f1f08101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.872+0000 7f1f0d94b700 1 -- 172.21.15.132:0/262305197 >> 172.21.15.132:0/262305197 conn(0x7f1f080788f0 msgr2=0x7f1f08078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:48.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.872+0000 7f1f0d94b700 1 -- 172.21.15.132:0/262305197 shutdown_connections 2024-04-23T18:34:48.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.872+0000 7f1f0d94b700 1 -- 172.21.15.132:0/262305197 wait complete. 2024-04-23T18:34:48.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.873+0000 7f1f0d94b700 1 Processor -- start 2024-04-23T18:34:48.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.874+0000 7f1f0d94b700 1 -- start start 2024-04-23T18:34:48.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.874+0000 7f1f0d94b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f08101940 0x7f1f0819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:48.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.874+0000 7f1f06ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f08101940 0x7f1f0819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:48.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.874+0000 7f1f0d94b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f08102300 0x7f1f0819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:48.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.874+0000 7f1f0d94b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f08107990 0x7f1f0819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:48.867 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.874+0000 7f1f0d94b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08112c40 con 0x7f1f08101940 2024-04-23T18:34:48.867 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.874+0000 7f1f06ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f08101940 0x7f1f0819b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53100/0 (socket says 172.21.15.132:53100) 2024-04-23T18:34:48.867 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.874+0000 7f1f06ffd700 1 -- 172.21.15.132:0/3295030521 learned_addr learned my addr 172.21.15.132:0/3295030521 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:48.867 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.874+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08112ac0 con 0x7f1f08102300 2024-04-23T18:34:48.867 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.874+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08112dc0 con 0x7f1f08107990 2024-04-23T18:34:48.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.875+0000 7f1f077fe700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f08107990 0x7f1f0819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:48.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.875+0000 7f1f06ffd700 1 -- 172.21.15.132:0/3295030521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f08107990 msgr2=0x7f1f0819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:48.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.875+0000 7f1f06ffd700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f08107990 0x7f1f0819fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.875+0000 7f1f06ffd700 1 -- 172.21.15.132:0/3295030521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f08102300 msgr2=0x7f1f0819b790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:34:48.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.875+0000 7f1f06ffd700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f08102300 0x7f1f0819b790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.875+0000 7f1f06ffd700 1 -- 172.21.15.132:0/3295030521 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f081a0260 con 0x7f1f08101940 2024-04-23T18:34:48.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.875+0000 7f1f06ffd700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f08101940 0x7f1f0819b250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800d980 tx=0x7f1ef800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:48.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.875+0000 7f1f0c949700 1 -- 172.21.15.132:0/3295030521 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8016070 con 0x7f1f08101940 2024-04-23T18:34:48.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.875+0000 7f1f0c949700 1 -- 172.21.15.132:0/3295030521 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef8004ce0 con 0x7f1f08101940 2024-04-23T18:34:48.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.875+0000 7f1f0c949700 1 -- 172.21.15.132:0/3295030521 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8005440 con 0x7f1f08101940 2024-04-23T18:34:48.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.875+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f081a0550 con 0x7f1f08101940 2024-04-23T18:34:48.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.876+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f081a7dd0 con 0x7f1f08101940 2024-04-23T18:34:48.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.877+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f08100210 con 0x7f1f08101940 2024-04-23T18:34:48.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.879+0000 7f1f0c949700 1 -- 172.21.15.132:0/3295030521 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1ef80055e0 con 0x7f1f08101940 2024-04-23T18:34:48.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.880+0000 7f1f0c949700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ef00a3d50 0x7f1ef00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:48.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.880+0000 7f1f0c949700 1 -- 172.21.15.132:0/3295030521 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f1ef80d06f0 con 0x7f1f08101940 2024-04-23T18:34:48.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.880+0000 7f1f067fc700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ef00a3d50 0x7f1ef00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:48.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.881+0000 7f1f0c949700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121] conn(0x7f1ef00ad530 0x7f1ef00af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:48.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.881+0000 7f1f0c949700 1 -- 172.21.15.132:0/3295030521 --> [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ef00b0000 con 0x7f1ef00ad530 2024-04-23T18:34:48.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.881+0000 7f1f0c949700 1 -- 172.21.15.132:0/3295030521 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f1ef80d0bd0 con 0x7f1f08101940 2024-04-23T18:34:48.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.881+0000 7f1f077fe700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121] conn(0x7f1ef00ad530 0x7f1ef00af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:48.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.881+0000 7f1f067fc700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ef00a3d50 0x7f1ef00a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1f08106f30 tx=0x7f1efc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:48.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.881+0000 7f1f077fe700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121] conn(0x7f1ef00ad530 0x7f1ef00af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:48.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.882+0000 7f1f0c949700 1 -- 172.21.15.132:0/3295030521 <== osd.4 v2:172.21.15.148:6808/1930596121 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ef00b0000 con 0x7f1ef00ad530 2024-04-23T18:34:48.896 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:48.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.910+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 --> [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f08103060 con 0x7f1ef00ad530 2024-04-23T18:34:48.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.911+0000 7f1f0c949700 1 -- 172.21.15.132:0/3295030521 <== osd.4 v2:172.21.15.148:6808/1930596121 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1f08103060 con 0x7f1ef00ad530 2024-04-23T18:34:48.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.911+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 >> [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121] conn(0x7f1ef00ad530 msgr2=0x7f1ef00af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:48.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.911+0000 7f1f0d94b700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121] conn(0x7f1ef00ad530 0x7f1ef00af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.912+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ef00a3d50 msgr2=0x7f1ef00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:48.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.912+0000 7f1f0d94b700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ef00a3d50 0x7f1ef00a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1f08106f30 tx=0x7f1efc009260 comp rx=0 tx=0).stop 2024-04-23T18:34:48.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.912+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f08101940 msgr2=0x7f1f0819b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:48.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.912+0000 7f1f0d94b700 1 --2- 172.21.15.132:0/3295030521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f08101940 0x7f1f0819b250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800d980 tx=0x7f1ef800dd40 comp rx=0 tx=0).stop 2024-04-23T18:34:48.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.912+0000 7f1f077fe700 1 -- 172.21.15.132:0/3295030521 reap_dead start 2024-04-23T18:34:48.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.912+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 shutdown_connections 2024-04-23T18:34:48.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.912+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 >> 172.21.15.132:0/3295030521 conn(0x7f1f080788f0 msgr2=0x7f1f080ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:48.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.912+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 shutdown_connections 2024-04-23T18:34:48.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:48.912+0000 7f1f0d94b700 1 -- 172.21.15.132:0/3295030521 wait complete. 2024-04-23T18:34:48.923 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:49.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:49 smithi148 ceph-mon[32311]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-23T18:34:49.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:49 smithi157 ceph-mon[32580]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-23T18:34:49.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:49 smithi132 ceph-mon[29178]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-23T18:34:49.854 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:51.435 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:51 smithi132 ceph-mon[29178]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-23T18:34:51.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:51 smithi148 ceph-mon[32311]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-23T18:34:51.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:51 smithi157 ceph-mon[32580]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-23T18:34:52.031 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:52.031 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:53.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:53 smithi132 ceph-mon[29178]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:34:53.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:53 smithi148 ceph-mon[32311]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:34:53.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:53 smithi157 ceph-mon[32580]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:34:54.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-23T18:34:54.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-23T18:34:54.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-23T18:34:54.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-04-23T18:34:54.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-23T18:34:54.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]: dispatch 2024-04-23T18:34:54.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 8]}]: dispatch 2024-04-23T18:34:54.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-04-23T18:34:54.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-23T18:34:54.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-23T18:34:54.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-23T18:34:54.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-23T18:34:54.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-04-23T18:34:54.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-23T18:34:54.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]: dispatch 2024-04-23T18:34:54.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 8]}]: dispatch 2024-04-23T18:34:54.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-04-23T18:34:54.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-23T18:34:54.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-23T18:34:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-23T18:34:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-23T18:34:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-04-23T18:34:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-23T18:34:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]: dispatch 2024-04-23T18:34:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 8]}]: dispatch 2024-04-23T18:34:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-04-23T18:34:54.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-23T18:34:55.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:55 smithi148 ceph-mon[32311]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:34:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-23T18:34:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-23T18:34:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-23T18:34:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-04-23T18:34:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-23T18:34:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]': finished 2024-04-23T18:34:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 8]}]': finished 2024-04-23T18:34:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-04-23T18:34:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-23T18:34:55.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:55 smithi148 ceph-mon[32311]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T18:34:55.516 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:55 smithi132 ceph-mon[29178]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:34:55.516 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-23T18:34:55.516 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-23T18:34:55.516 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-23T18:34:55.516 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-04-23T18:34:55.516 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-23T18:34:55.516 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]': finished 2024-04-23T18:34:55.516 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 8]}]': finished 2024-04-23T18:34:55.516 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-04-23T18:34:55.516 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-23T18:34:55.516 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:55 smithi132 ceph-mon[29178]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T18:34:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:55 smithi157 ceph-mon[32580]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:34:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-23T18:34:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-23T18:34:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-23T18:34:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-04-23T18:34:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-23T18:34:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]': finished 2024-04-23T18:34:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 8]}]': finished 2024-04-23T18:34:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-04-23T18:34:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-23T18:34:55.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:55 smithi157 ceph-mon[32580]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T18:34:56.143 INFO:teuthology.orchestra.run.smithi132.stdout:141733920827 2024-04-23T18:34:56.143 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-23T18:34:56.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:56 smithi132 ceph-mon[29178]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T18:34:56.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:56 smithi148 ceph-mon[32311]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T18:34:56.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:56 smithi157 ceph-mon[32580]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T18:34:57.512 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:57 smithi132 ceph-mon[29178]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-23T18:34:57.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:57 smithi148 ceph-mon[32311]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-23T18:34:57.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:57 smithi157 ceph-mon[32580]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-23T18:34:57.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.623+0000 7f621a78a700 1 -- 172.21.15.132:0/3717364287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214102300 msgr2=0x7f621410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:57.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.623+0000 7f621a78a700 1 --2- 172.21.15.132:0/3717364287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214102300 0x7f621410e7f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f620c00b0a0 tx=0x7f620c00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.623+0000 7f621a78a700 1 -- 172.21.15.132:0/3717364287 shutdown_connections 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.623+0000 7f621a78a700 1 --2- 172.21.15.132:0/3717364287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214102300 0x7f621410e7f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.623+0000 7f621a78a700 1 --2- 172.21.15.132:0/3717364287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6214107990 0x7f6214107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.623+0000 7f621a78a700 1 --2- 172.21.15.132:0/3717364287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6214101940 0x7f6214101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.623+0000 7f621a78a700 1 -- 172.21.15.132:0/3717364287 >> 172.21.15.132:0/3717364287 conn(0x7f62140788f0 msgr2=0x7f6214078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.623+0000 7f621a78a700 1 -- 172.21.15.132:0/3717364287 shutdown_connections 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.623+0000 7f621a78a700 1 -- 172.21.15.132:0/3717364287 wait complete. 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f621a78a700 1 Processor -- start 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f621a78a700 1 -- start start 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f621a78a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6214101940 0x7f6214199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f621a78a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214102300 0x7f6214199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f621a78a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6214107990 0x7f621419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f621a78a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6214111bb0 con 0x7f6214102300 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f621a78a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6214111a30 con 0x7f6214107990 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f621a78a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6214111d30 con 0x7f6214101940 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f62137fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214102300 0x7f6214199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f62137fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214102300 0x7f6214199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48246/0 (socket says 172.21.15.132:48246) 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f62137fe700 1 -- 172.21.15.132:0/2676712335 learned_addr learned my addr 172.21.15.132:0/2676712335 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f6213fff700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6214101940 0x7f6214199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.624+0000 7f6218d27700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6214107990 0x7f621419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.625+0000 7f62137fe700 1 -- 172.21.15.132:0/2676712335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6214101940 msgr2=0x7f6214199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.625+0000 7f62137fe700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6214101940 0x7f6214199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.625+0000 7f62137fe700 1 -- 172.21.15.132:0/2676712335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6214107990 msgr2=0x7f621419d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.625+0000 7f62137fe700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6214107990 0x7f621419d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.625+0000 7f62137fe700 1 -- 172.21.15.132:0/2676712335 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621419e130 con 0x7f6214102300 2024-04-23T18:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.625+0000 7f6218d27700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6214107990 0x7f621419d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:57.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.625+0000 7f62137fe700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214102300 0x7f6214199660 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f620800c920 tx=0x7f620800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:57.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.625+0000 7f62117fa700 1 -- 172.21.15.132:0/2676712335 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208007b00 con 0x7f6214102300 2024-04-23T18:34:57.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.625+0000 7f621a78a700 1 -- 172.21.15.132:0/2676712335 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621419e420 con 0x7f6214102300 2024-04-23T18:34:57.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.625+0000 7f62117fa700 1 -- 172.21.15.132:0/2676712335 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6208004ce0 con 0x7f6214102300 2024-04-23T18:34:57.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.625+0000 7f62117fa700 1 -- 172.21.15.132:0/2676712335 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208005500 con 0x7f6214102300 2024-04-23T18:34:57.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.625+0000 7f621a78a700 1 -- 172.21.15.132:0/2676712335 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62141a5ca0 con 0x7f6214102300 2024-04-23T18:34:57.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.626+0000 7f621a78a700 1 -- 172.21.15.132:0/2676712335 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6214069730 con 0x7f6214102300 2024-04-23T18:34:57.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.627+0000 7f62117fa700 1 -- 172.21.15.132:0/2676712335 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6208007ca0 con 0x7f6214102300 2024-04-23T18:34:57.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.628+0000 7f62117fa700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f61fc0a3cd0 0x7f61fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:57.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.628+0000 7f6213fff700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f61fc0a3cd0 0x7f61fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:57.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.628+0000 7f62117fa700 1 -- 172.21.15.132:0/2676712335 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f62080d0150 con 0x7f6214102300 2024-04-23T18:34:57.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.628+0000 7f6213fff700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f61fc0a3cd0 0x7f61fc0a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6204009a10 tx=0x7f6204009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:57.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.632+0000 7f62117fa700 1 -- 172.21.15.132:0/2676712335 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6208014030 con 0x7f6214102300 2024-04-23T18:34:57.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.845+0000 7f621a78a700 1 -- 172.21.15.132:0/2676712335 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f621419a480 con 0x7f6214102300 2024-04-23T18:34:57.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.846+0000 7f62117fa700 1 -- 172.21.15.132:0/2676712335 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f620809c9c0 con 0x7f6214102300 2024-04-23T18:34:57.838 INFO:teuthology.orchestra.run.smithi132.stdout:77309411399 2024-04-23T18:34:57.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.848+0000 7f621a78a700 1 -- 172.21.15.132:0/2676712335 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f61fc0a3cd0 msgr2=0x7f61fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:57.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.848+0000 7f621a78a700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f61fc0a3cd0 0x7f61fc0a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6204009a10 tx=0x7f6204009190 comp rx=0 tx=0).stop 2024-04-23T18:34:57.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.848+0000 7f621a78a700 1 -- 172.21.15.132:0/2676712335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214102300 msgr2=0x7f6214199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:57.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.848+0000 7f621a78a700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214102300 0x7f6214199660 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f620800c920 tx=0x7f620800cc30 comp rx=0 tx=0).stop 2024-04-23T18:34:57.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.848+0000 7f621a78a700 1 -- 172.21.15.132:0/2676712335 shutdown_connections 2024-04-23T18:34:57.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.848+0000 7f621a78a700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6214107990 0x7f621419d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:57.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.848+0000 7f621a78a700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6214101940 0x7f6214199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:57.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.848+0000 7f621a78a700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f61fc0a3cd0 0x7f61fc0a6190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:57.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.848+0000 7f621a78a700 1 --2- 172.21.15.132:0/2676712335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214102300 0x7f6214199660 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:57.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.848+0000 7f621a78a700 1 -- 172.21.15.132:0/2676712335 >> 172.21.15.132:0/2676712335 conn(0x7f62140788f0 msgr2=0x7f621406aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:57.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.848+0000 7f621a78a700 1 -- 172.21.15.132:0/2676712335 shutdown_connections 2024-04-23T18:34:57.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:57.848+0000 7f621a78a700 1 -- 172.21.15.132:0/2676712335 wait complete. 2024-04-23T18:34:58.473 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:58.474 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:58.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:58 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2676712335' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T18:34:58.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:58 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2676712335' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T18:34:58.572 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:58 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2676712335' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T18:34:59.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.038+0000 7f8748984700 1 -- 172.21.15.132:0/3326188966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87441019d0 msgr2=0x7f874410f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:59.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.038+0000 7f8748984700 1 --2- 172.21.15.132:0/3326188966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87441019d0 0x7f874410f910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f873c00b0a0 tx=0x7f873c00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.039+0000 7f8748984700 1 -- 172.21.15.132:0/3326188966 shutdown_connections 2024-04-23T18:34:59.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.039+0000 7f8748984700 1 --2- 172.21.15.132:0/3326188966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87441019d0 0x7f874410f910 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.039+0000 7f8748984700 1 --2- 172.21.15.132:0/3326188966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8744108ad0 0x7f8744108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.039+0000 7f8748984700 1 --2- 172.21.15.132:0/3326188966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8744101030 0x7f8744101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.039+0000 7f8748984700 1 -- 172.21.15.132:0/3326188966 >> 172.21.15.132:0/3326188966 conn(0x7f8744077fb0 msgr2=0x7f87440ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:59.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.039+0000 7f8748984700 1 -- 172.21.15.132:0/3326188966 shutdown_connections 2024-04-23T18:34:59.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.039+0000 7f8748984700 1 -- 172.21.15.132:0/3326188966 wait complete. 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.039+0000 7f8748984700 1 Processor -- start 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.039+0000 7f8748984700 1 -- start start 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8748984700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8744101030 0x7f8744199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8748984700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87441019d0 0x7f87441995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8748984700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8744108ad0 0x7f874419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8748984700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8744112be0 con 0x7f8744108ad0 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8748984700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8744112a60 con 0x7f8744101030 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8748984700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8744112d60 con 0x7f87441019d0 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8742d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8744108ad0 0x7f874419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8742d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8744108ad0 0x7f874419d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48272/0 (socket says 172.21.15.132:48272) 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8742d9d700 1 -- 172.21.15.132:0/2296043247 learned_addr learned my addr 172.21.15.132:0/2296043247 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f874259c700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8744101030 0x7f8744199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:59.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8741d9b700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87441019d0 0x7f87441995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:59.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8742d9d700 1 -- 172.21.15.132:0/2296043247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87441019d0 msgr2=0x7f87441995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:59.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8742d9d700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87441019d0 0x7f87441995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8742d9d700 1 -- 172.21.15.132:0/2296043247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8744101030 msgr2=0x7f8744199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:59.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8742d9d700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8744101030 0x7f8744199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8742d9d700 1 -- 172.21.15.132:0/2296043247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874419e0a0 con 0x7f8744108ad0 2024-04-23T18:34:59.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f8742d9d700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8744108ad0 0x7f874419d960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f873c00b830 tx=0x7f873c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:59.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.040+0000 7f87337fe700 1 -- 172.21.15.132:0/2296043247 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c021070 con 0x7f8744108ad0 2024-04-23T18:34:59.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.041+0000 7f8748984700 1 -- 172.21.15.132:0/2296043247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874419e330 con 0x7f8744108ad0 2024-04-23T18:34:59.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.041+0000 7f87337fe700 1 -- 172.21.15.132:0/2296043247 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f873c007e30 con 0x7f8744108ad0 2024-04-23T18:34:59.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.041+0000 7f87337fe700 1 -- 172.21.15.132:0/2296043247 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c0047d0 con 0x7f8744108ad0 2024-04-23T18:34:59.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.041+0000 7f8748984700 1 -- 172.21.15.132:0/2296043247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87441a5c10 con 0x7f8744108ad0 2024-04-23T18:34:59.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.042+0000 7f8748984700 1 -- 172.21.15.132:0/2296043247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8724005330 con 0x7f8744108ad0 2024-04-23T18:34:59.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.043+0000 7f87337fe700 1 -- 172.21.15.132:0/2296043247 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f873c004020 con 0x7f8744108ad0 2024-04-23T18:34:59.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.043+0000 7f87337fe700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f872c0a3cd0 0x7f872c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:59.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.044+0000 7f874259c700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f872c0a3cd0 0x7f872c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:59.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.044+0000 7f87337fe700 1 -- 172.21.15.132:0/2296043247 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f873c00f020 con 0x7f8744108ad0 2024-04-23T18:34:59.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.044+0000 7f874259c700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f872c0a3cd0 0x7f872c0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8734005950 tx=0x7f873400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:59.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.050+0000 7f87337fe700 1 -- 172.21.15.132:0/2296043247 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f873c02a020 con 0x7f8744108ad0 2024-04-23T18:34:59.261 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.269+0000 7f8748984700 1 -- 172.21.15.132:0/2296043247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f87240058f0 con 0x7f8744108ad0 2024-04-23T18:34:59.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.278+0000 7f87337fe700 1 -- 172.21.15.132:0/2296043247 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f873c09e2d0 con 0x7f8744108ad0 2024-04-23T18:34:59.270 INFO:teuthology.orchestra.run.smithi132.stdout:197568495667 2024-04-23T18:34:59.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.264+0000 7f1e867b1700 1 -- 172.21.15.132:0/1159854438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e8006afe0 msgr2=0x7f1e8006b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:59.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.264+0000 7f1e867b1700 1 --2- 172.21.15.132:0/1159854438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e8006afe0 0x7f1e8006b440 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009990 tx=0x7f1e68009ca0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.264+0000 7f1e867b1700 1 -- 172.21.15.132:0/1159854438 shutdown_connections 2024-04-23T18:34:59.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.264+0000 7f1e867b1700 1 --2- 172.21.15.132:0/1159854438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e8006b980 0x7f1e8010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.264+0000 7f1e867b1700 1 --2- 172.21.15.132:0/1159854438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e80108a20 0x7f1e80108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.264+0000 7f1e867b1700 1 --2- 172.21.15.132:0/1159854438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e8006afe0 0x7f1e8006b440 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.264+0000 7f1e867b1700 1 -- 172.21.15.132:0/1159854438 >> 172.21.15.132:0/1159854438 conn(0x7f1e800788b0 msgr2=0x7f1e80078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:59.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.264+0000 7f1e867b1700 1 -- 172.21.15.132:0/1159854438 shutdown_connections 2024-04-23T18:34:59.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.264+0000 7f1e867b1700 1 -- 172.21.15.132:0/1159854438 wait complete. 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.264+0000 7f1e867b1700 1 Processor -- start 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.264+0000 7f1e867b1700 1 -- start start 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e867b1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e8006afe0 0x7f1e801990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e867b1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8006b980 0x7f1e80199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e867b1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e80108a20 0x7f1e8019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e867b1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80112be0 con 0x7f1e80108a20 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e867b1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80112a60 con 0x7f1e8006afe0 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e867b1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80112d60 con 0x7f1e8006b980 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e84d4e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e80108a20 0x7f1e8019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e84d4e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e80108a20 0x7f1e8019d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48284/0 (socket says 172.21.15.132:48284) 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e84d4e700 1 -- 172.21.15.132:0/1746158787 learned_addr learned my addr 172.21.15.132:0/1746158787 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e7ffff700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e8006afe0 0x7f1e801990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e777fe700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8006b980 0x7f1e80199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:59.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e84d4e700 1 -- 172.21.15.132:0/1746158787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8006b980 msgr2=0x7f1e80199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e84d4e700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8006b980 0x7f1e80199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e84d4e700 1 -- 172.21.15.132:0/1746158787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e8006afe0 msgr2=0x7f1e801990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e84d4e700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e8006afe0 0x7f1e801990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e84d4e700 1 -- 172.21.15.132:0/1746158787 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8019e100 con 0x7f1e80108a20 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e7ffff700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e8006afe0 0x7f1e801990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.265+0000 7f1e84d4e700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e80108a20 0x7f1e8019d9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800ef30 tx=0x7f1e7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.266+0000 7f1e7dffb700 1 -- 172.21.15.132:0/1746158787 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e780092e0 con 0x7f1e80108a20 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.266+0000 7f1e7dffb700 1 -- 172.21.15.132:0/1746158787 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e78004bb0 con 0x7f1e80108a20 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.266+0000 7f1e867b1700 1 -- 172.21.15.132:0/1746158787 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8019e390 con 0x7f1e80108a20 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.266+0000 7f1e7dffb700 1 -- 172.21.15.132:0/1746158787 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78004020 con 0x7f1e80108a20 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.266+0000 7f1e867b1700 1 -- 172.21.15.132:0/1746158787 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e801a5c70 con 0x7f1e80108a20 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.267+0000 7f1e867b1700 1 -- 172.21.15.132:0/1746158787 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e80104160 con 0x7f1e80108a20 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.268+0000 7f1e7dffb700 1 -- 172.21.15.132:0/1746158787 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1e7801b030 con 0x7f1e80108a20 2024-04-23T18:34:59.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.269+0000 7f1e7dffb700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1e6c0a3d50 0x7f1e6c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:59.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.269+0000 7f1e7ffff700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1e6c0a3d50 0x7f1e6c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:59.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.269+0000 7f1e7dffb700 1 -- 172.21.15.132:0/1746158787 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f1e780cfc60 con 0x7f1e80108a20 2024-04-23T18:34:59.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.269+0000 7f1e7ffff700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1e6c0a3d50 0x7f1e6c0a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009fd0 tx=0x7f1e70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:59.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.278+0000 7f1e7dffb700 1 -- 172.21.15.132:0/1746158787 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e78010040 con 0x7f1e80108a20 2024-04-23T18:34:59.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.280+0000 7f8748984700 1 -- 172.21.15.132:0/2296043247 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f872c0a3cd0 msgr2=0x7f872c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:59.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.280+0000 7f8748984700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f872c0a3cd0 0x7f872c0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8734005950 tx=0x7f873400a300 comp rx=0 tx=0).stop 2024-04-23T18:34:59.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.280+0000 7f8748984700 1 -- 172.21.15.132:0/2296043247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8744108ad0 msgr2=0x7f874419d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:59.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.280+0000 7f8748984700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8744108ad0 0x7f874419d960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f873c00b830 tx=0x7f873c004ce0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.280+0000 7f8748984700 1 -- 172.21.15.132:0/2296043247 shutdown_connections 2024-04-23T18:34:59.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.280+0000 7f8748984700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f872c0a3cd0 0x7f872c0a6190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.280+0000 7f8748984700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8744108ad0 0x7f874419d960 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.280+0000 7f8748984700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8744101030 0x7f8744199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.280+0000 7f8748984700 1 --2- 172.21.15.132:0/2296043247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87441019d0 0x7f87441995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.280+0000 7f8748984700 1 -- 172.21.15.132:0/2296043247 >> 172.21.15.132:0/2296043247 conn(0x7f8744077fb0 msgr2=0x7f87440ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:59.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.281+0000 7f8748984700 1 -- 172.21.15.132:0/2296043247 shutdown_connections 2024-04-23T18:34:59.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.281+0000 7f8748984700 1 -- 172.21.15.132:0/2296043247 wait complete. 2024-04-23T18:34:59.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.489+0000 7f1e867b1700 1 -- 172.21.15.132:0/1746158787 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f1e8002df80 con 0x7f1e80108a20 2024-04-23T18:34:59.482 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:59 smithi132 ceph-mon[29178]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:34:59.482 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:34:59 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:34:59.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.490+0000 7f1e7dffb700 1 -- 172.21.15.132:0/1746158787 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1e7801b320 con 0x7f1e80108a20 2024-04-23T18:34:59.482 INFO:teuthology.orchestra.run.smithi132.stdout:347892351001 2024-04-23T18:34:59.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.492+0000 7f1e867b1700 1 -- 172.21.15.132:0/1746158787 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1e6c0a3d50 msgr2=0x7f1e6c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:59.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.492+0000 7f1e867b1700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1e6c0a3d50 0x7f1e6c0a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009fd0 tx=0x7f1e70009260 comp rx=0 tx=0).stop 2024-04-23T18:34:59.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.492+0000 7f1e867b1700 1 -- 172.21.15.132:0/1746158787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e80108a20 msgr2=0x7f1e8019d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:59.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.492+0000 7f1e867b1700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e80108a20 0x7f1e8019d9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800ef30 tx=0x7f1e7800c3a0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.492+0000 7f1e867b1700 1 -- 172.21.15.132:0/1746158787 shutdown_connections 2024-04-23T18:34:59.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.492+0000 7f1e867b1700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1e6c0a3d50 0x7f1e6c0a6210 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.492+0000 7f1e867b1700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e80108a20 0x7f1e8019d9c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.492+0000 7f1e867b1700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e8006afe0 0x7f1e801990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.492+0000 7f1e867b1700 1 --2- 172.21.15.132:0/1746158787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8006b980 0x7f1e80199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:59.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.492+0000 7f1e867b1700 1 -- 172.21.15.132:0/1746158787 >> 172.21.15.132:0/1746158787 conn(0x7f1e800788b0 msgr2=0x7f1e800ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:59.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.492+0000 7f1e867b1700 1 -- 172.21.15.132:0/1746158787 shutdown_connections 2024-04-23T18:34:59.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:34:59.492+0000 7f1e867b1700 1 -- 172.21.15.132:0/1746158787 wait complete. 2024-04-23T18:34:59.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:59 smithi157 ceph-mon[32580]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:34:59.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:34:59 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:34:59.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:59 smithi148 ceph-mon[32311]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:34:59.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:34:59 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:34:59.898 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411399 for osd.2 2024-04-23T18:34:59.898 DEBUG:teuthology.parallel:result is None 2024-04-23T18:35:00.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:00 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2296043247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T18:35:00.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:00 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1746158787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T18:35:00.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:00 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2296043247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T18:35:00.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:00 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1746158787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T18:35:00.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:00 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2296043247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T18:35:00.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:00 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1746158787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T18:35:01.405 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:01 smithi132 ceph-mon[29178]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:35:01.405 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:01.405 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:01.405 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:01.405 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:01 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:01.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:01 smithi157 ceph-mon[32580]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:35:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:01.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:01 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:01.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:01 smithi148 ceph-mon[32311]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:35:01.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:01.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:01.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:01.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:01 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:02.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:02 smithi132 ceph-mon[29178]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-04-23T18:35:02.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:02 smithi157 ceph-mon[32580]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-04-23T18:35:02.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.603+0000 7ffa407d0700 1 -- 172.21.15.132:0/2388140421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa381034b0 msgr2=0x7ffa3810f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:02.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.603+0000 7ffa407d0700 1 --2- 172.21.15.132:0/2388140421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa381034b0 0x7ffa3810f8e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000b0a0 tx=0x7ffa3000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:35:02.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 -- 172.21.15.132:0/2388140421 shutdown_connections 2024-04-23T18:35:02.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 --2- 172.21.15.132:0/2388140421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa381034b0 0x7ffa3810f8e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:02.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 --2- 172.21.15.132:0/2388140421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa38108a80 0x7ffa38108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:02.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 --2- 172.21.15.132:0/2388140421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa38102af0 0x7ffa38102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:02.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 -- 172.21.15.132:0/2388140421 >> 172.21.15.132:0/2388140421 conn(0x7ffa38079b20 msgr2=0x7ffa38079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:02.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 -- 172.21.15.132:0/2388140421 shutdown_connections 2024-04-23T18:35:02.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 -- 172.21.15.132:0/2388140421 wait complete. 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 Processor -- start 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 -- start start 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa38102af0 0x7ffa38074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa381034b0 0x7ffa38075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa38108a80 0x7ffa380757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ffa38112c10 con 0x7ffa381034b0 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa38112a90 con 0x7ffa38102af0 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.604+0000 7ffa407d0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa38112d90 con 0x7ffa38108a80 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa3dd6b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa381034b0 0x7ffa38075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa3dd6b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa381034b0 0x7ffa38075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54260/0 (socket says 172.21.15.132:54260) 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa3dd6b700 1 -- 172.21.15.132:0/3603934523 learned_addr learned my addr 172.21.15.132:0/3603934523 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa3e56c700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa38102af0 0x7ffa38074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa3ed6d700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa38108a80 0x7ffa380757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:02.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa3dd6b700 1 -- 172.21.15.132:0/3603934523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa38108a80 msgr2=0x7ffa380757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa3dd6b700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa38108a80 0x7ffa380757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa3dd6b700 1 -- 172.21.15.132:0/3603934523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa38102af0 msgr2=0x7ffa38074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa3dd6b700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa38102af0 0x7ffa38074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa3dd6b700 1 -- 172.21.15.132:0/3603934523 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa381a75d0 con 0x7ffa381034b0 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa3dd6b700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa381034b0 0x7ffa38075280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800e9a0 tx=0x7ffa2800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa2f7fe700 1 -- 172.21.15.132:0/3603934523 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2800cbd0 con 0x7ffa381034b0 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa2f7fe700 1 -- 172.21.15.132:0/3603934523 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa2800cd70 con 0x7ffa381034b0 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa407d0700 1 -- 172.21.15.132:0/3603934523 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa381a78c0 con 0x7ffa381034b0 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa2f7fe700 1 -- 172.21.15.132:0/3603934523 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa28018680 con 0x7ffa381034b0 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.605+0000 7ffa407d0700 1 -- 172.21.15.132:0/3603934523 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa381a7e00 con 0x7ffa381034b0 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.606+0000 7ffa407d0700 1 -- 172.21.15.132:0/3603934523 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa38069730 con 0x7ffa381034b0 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.608+0000 7ffa2f7fe700 1 -- 172.21.15.132:0/3603934523 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ffa28010970 con 0x7ffa381034b0 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.608+0000 7ffa2f7fe700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ffa240a3cd0 0x7ffa240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.609+0000 7ffa3e56c700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ffa240a3cd0 0x7ffa240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:02.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.609+0000 7ffa2f7fe700 1 -- 172.21.15.132:0/3603934523 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ffa280d0e10 con 0x7ffa381034b0 2024-04-23T18:35:02.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.609+0000 7ffa3e56c700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ffa240a3cd0 0x7ffa240a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005950 tx=0x7ffa340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:02.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.612+0000 7ffa2f7fe700 1 -- 172.21.15.132:0/3603934523 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa2809d620 con 0x7ffa381034b0 2024-04-23T18:35:02.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:02 smithi148 ceph-mon[32311]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-04-23T18:35:02.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.820+0000 7ffa407d0700 1 -- 172.21.15.132:0/3603934523 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ffa3802e140 con 0x7ffa381034b0 2024-04-23T18:35:02.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.821+0000 7ffa2f7fe700 1 -- 172.21.15.132:0/3603934523 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffa2809cf20 con 0x7ffa381034b0 2024-04-23T18:35:02.814 INFO:teuthology.orchestra.run.smithi132.stdout:223338299438 2024-04-23T18:35:02.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.823+0000 7ffa407d0700 1 -- 172.21.15.132:0/3603934523 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ffa240a3cd0 msgr2=0x7ffa240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:02.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.823+0000 7ffa407d0700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ffa240a3cd0 0x7ffa240a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005950 tx=0x7ffa340058e0 comp rx=0 tx=0).stop 2024-04-23T18:35:02.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.823+0000 7ffa407d0700 1 -- 172.21.15.132:0/3603934523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa381034b0 msgr2=0x7ffa38075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:02.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.823+0000 7ffa407d0700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa381034b0 0x7ffa38075280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800e9a0 tx=0x7ffa2800ed60 comp rx=0 tx=0).stop 2024-04-23T18:35:02.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.823+0000 7ffa407d0700 1 -- 172.21.15.132:0/3603934523 shutdown_connections 2024-04-23T18:35:02.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.823+0000 7ffa407d0700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa38108a80 0x7ffa380757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:02.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.823+0000 7ffa407d0700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa38102af0 0x7ffa38074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:02.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.823+0000 7ffa407d0700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ffa240a3cd0 0x7ffa240a6190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:02.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.823+0000 7ffa407d0700 1 --2- 172.21.15.132:0/3603934523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa381034b0 0x7ffa38075280 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:02.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.823+0000 7ffa407d0700 1 -- 172.21.15.132:0/3603934523 >> 172.21.15.132:0/3603934523 conn(0x7ffa38079b20 msgr2=0x7ffa381017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:02.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.823+0000 7ffa407d0700 1 -- 172.21.15.132:0/3603934523 shutdown_connections 2024-04-23T18:35:02.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:02.823+0000 7ffa407d0700 1 -- 172.21.15.132:0/3603934523 wait complete. 2024-04-23T18:35:03.037 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350998 got 347892351001 for osd.11 2024-04-23T18:35:03.037 DEBUG:teuthology.parallel:result is None 2024-04-23T18:35:03.454 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:03 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3603934523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T18:35:03.482 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495664 got 197568495667 for osd.6 2024-04-23T18:35:03.482 DEBUG:teuthology.parallel:result is None 2024-04-23T18:35:03.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:03 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3603934523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T18:35:03.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:03 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3603934523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T18:35:04.096 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299434 got 223338299438 for osd.7 2024-04-23T18:35:04.096 DEBUG:teuthology.parallel:result is None 2024-04-23T18:35:04.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:04 smithi132 ceph-mon[29178]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:35:04.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:04 smithi148 ceph-mon[32311]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:35:04.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:04 smithi157 ceph-mon[32580]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:35:05.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.893+0000 7f0647c9f700 1 -- 172.21.15.132:0/758560020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640102300 msgr2=0x7f064010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:05.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.893+0000 7f0647c9f700 1 --2- 172.21.15.132:0/758560020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640102300 0x7f064010e7f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f063800b0a0 tx=0x7f063800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:35:05.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.893+0000 7f0647c9f700 1 -- 172.21.15.132:0/758560020 shutdown_connections 2024-04-23T18:35:05.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.893+0000 7f0647c9f700 1 --2- 172.21.15.132:0/758560020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640102300 0x7f064010e7f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:05.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.893+0000 7f0647c9f700 1 --2- 172.21.15.132:0/758560020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0640107990 0x7f0640107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:05.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.893+0000 7f0647c9f700 1 --2- 172.21.15.132:0/758560020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0640101940 0x7f0640101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.893+0000 7f0647c9f700 1 -- 172.21.15.132:0/758560020 >> 172.21.15.132:0/758560020 conn(0x7f06400788f0 msgr2=0x7f0640078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.893+0000 7f0647c9f700 1 -- 172.21.15.132:0/758560020 shutdown_connections 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.893+0000 7f0647c9f700 1 -- 172.21.15.132:0/758560020 wait complete. 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f0647c9f700 1 Processor -- start 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f0647c9f700 1 -- start start 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f0647c9f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0640101940 0x7f0640199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f0647c9f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0640102300 0x7f064019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f0647c9f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640107990 0x7f06401a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f0647c9f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0640111b60 con 0x7f0640107990 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f0647c9f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f06401119e0 con 0x7f0640101940 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f0647c9f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0640111ce0 con 0x7f0640102300 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f064623c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640107990 0x7f06401a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f064623c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640107990 0x7f06401a4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54274/0 (socket says 172.21.15.132:54274) 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f064623c700 1 -- 172.21.15.132:0/1680919036 learned_addr learned my addr 172.21.15.132:0/1680919036 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:05.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f0645a3b700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0640101940 0x7f0640199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:05.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f064523a700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0640102300 0x7f064019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:05.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f064623c700 1 -- 172.21.15.132:0/1680919036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0640102300 msgr2=0x7f064019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:05.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f064623c700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0640102300 0x7f064019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:05.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f064623c700 1 -- 172.21.15.132:0/1680919036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0640101940 msgr2=0x7f0640199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:05.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f064623c700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0640101940 0x7f0640199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:05.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f064623c700 1 -- 172.21.15.132:0/1680919036 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06401a54c0 con 0x7f0640107990 2024-04-23T18:35:05.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.894+0000 7f064623c700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640107990 0x7f06401a4f80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f06380062a0 tx=0x7f0638004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:05.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.895+0000 7f0636ffd700 1 -- 172.21.15.132:0/1680919036 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0638021070 con 0x7f0640107990 2024-04-23T18:35:05.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.895+0000 7f0636ffd700 1 -- 172.21.15.132:0/1680919036 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0638007d00 con 0x7f0640107990 2024-04-23T18:35:05.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.895+0000 7f0647c9f700 1 -- 172.21.15.132:0/1680919036 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06401a5690 con 0x7f0640107990 2024-04-23T18:35:05.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.895+0000 7f0636ffd700 1 -- 172.21.15.132:0/1680919036 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063801d450 con 0x7f0640107990 2024-04-23T18:35:05.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.895+0000 7f0647c9f700 1 -- 172.21.15.132:0/1680919036 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06401a5b90 con 0x7f0640107990 2024-04-23T18:35:05.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.896+0000 7f0647c9f700 1 -- 172.21.15.132:0/1680919036 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0640109eb0 con 0x7f0640107990 2024-04-23T18:35:05.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.897+0000 7f0636ffd700 1 -- 172.21.15.132:0/1680919036 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0638004000 con 0x7f0640107990 2024-04-23T18:35:05.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.897+0000 7f0636ffd700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f062c0a3d50 0x7f062c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:05.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.897+0000 7f0645a3b700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f062c0a3d50 0x7f062c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:05.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.898+0000 7f0636ffd700 1 -- 172.21.15.132:0/1680919036 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f063800f020 con 0x7f0640107990 2024-04-23T18:35:05.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.898+0000 7f0645a3b700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f062c0a3d50 0x7f062c0a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f063c006fd0 tx=0x7f063c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:05.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:05.901+0000 7f0636ffd700 1 -- 172.21.15.132:0/1680919036 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f063802a020 con 0x7f0640107990 2024-04-23T18:35:06.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.076+0000 7f74a9539700 1 -- 172.21.15.132:0/797911536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a406a750 msgr2=0x7f74a406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:06.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.076+0000 7f74a9539700 1 --2- 172.21.15.132:0/797911536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a406a750 0x7f74a406abb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7494009990 tx=0x7f7494009ca0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.076+0000 7f74a9539700 1 -- 172.21.15.132:0/797911536 shutdown_connections 2024-04-23T18:35:06.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.076+0000 7f74a9539700 1 --2- 172.21.15.132:0/797911536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a406b0f0 0x7f74a410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.076+0000 7f74a9539700 1 --2- 172.21.15.132:0/797911536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4108a30 0x7f74a4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.076+0000 7f74a9539700 1 --2- 172.21.15.132:0/797911536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a406a750 0x7f74a406abb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.076+0000 7f74a9539700 1 -- 172.21.15.132:0/797911536 >> 172.21.15.132:0/797911536 conn(0x7f74a40788f0 msgr2=0x7f74a4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:06.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.076+0000 7f74a9539700 1 -- 172.21.15.132:0/797911536 shutdown_connections 2024-04-23T18:35:06.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.076+0000 7f74a9539700 1 -- 172.21.15.132:0/797911536 wait complete. 2024-04-23T18:35:06.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.077+0000 7f74a9539700 1 Processor -- start 2024-04-23T18:35:06.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.077+0000 7f74a9539700 1 -- start start 2024-04-23T18:35:06.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.077+0000 7f74a9539700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a406a750 0x7f74a4101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:06.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.077+0000 7f74a9539700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a406b0f0 0x7f74a4101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:06.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.077+0000 7f74a9539700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4108a30 0x7f74a4102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.077+0000 7f74a9539700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f74a4112bd0 con 0x7f74a406a750 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.077+0000 7f74a9539700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f74a4112a50 con 0x7f74a406b0f0 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.077+0000 7f74a9539700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f74a4112d50 con 0x7f74a4108a30 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.077+0000 7f74a2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a406a750 0x7f74a4101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.077+0000 7f74a27fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a406b0f0 0x7f74a4101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.077+0000 7f74a2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a406a750 0x7f74a4101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54286/0 (socket says 172.21.15.132:54286) 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.077+0000 7f74a2ffd700 1 -- 172.21.15.132:0/3669581246 learned_addr learned my addr 172.21.15.132:0/3669581246 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f74a37fe700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4108a30 0x7f74a4102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f74a2ffd700 1 -- 172.21.15.132:0/3669581246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4108a30 msgr2=0x7f74a4102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f74a2ffd700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4108a30 0x7f74a4102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f74a2ffd700 1 -- 172.21.15.132:0/3669581246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a406b0f0 msgr2=0x7f74a4101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f74a2ffd700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a406b0f0 0x7f74a4101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f74a2ffd700 1 -- 172.21.15.132:0/3669581246 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a41a7680 con 0x7f74a406a750 2024-04-23T18:35:06.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f74a27fc700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a406b0f0 0x7f74a4101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:35:06.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f74a2ffd700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a406a750 0x7f74a4101a10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f749800d980 tx=0x7f749800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:06.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f748bfff700 1 -- 172.21.15.132:0/3669581246 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498017070 con 0x7f74a406a750 2024-04-23T18:35:06.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f74a9539700 1 -- 172.21.15.132:0/3669581246 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a41a7970 con 0x7f74a406a750 2024-04-23T18:35:06.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f74a9539700 1 -- 172.21.15.132:0/3669581246 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a41a7e80 con 0x7f74a406a750 2024-04-23T18:35:06.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f748bfff700 1 -- 172.21.15.132:0/3669581246 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7498005020 con 0x7f74a406a750 2024-04-23T18:35:06.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.078+0000 7f748bfff700 1 -- 172.21.15.132:0/3669581246 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749800fa10 con 0x7f74a406a750 2024-04-23T18:35:06.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.079+0000 7f74a9539700 1 -- 172.21.15.132:0/3669581246 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7490005330 con 0x7f74a406a750 2024-04-23T18:35:06.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.081+0000 7f748bfff700 1 -- 172.21.15.132:0/3669581246 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7498004a30 con 0x7f74a406a750 2024-04-23T18:35:06.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.081+0000 7f748bfff700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f748c0a3d50 0x7f748c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:06.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.082+0000 7f74a27fc700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f748c0a3d50 0x7f748c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:06.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.082+0000 7f748bfff700 1 -- 172.21.15.132:0/3669581246 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f74980d05a0 con 0x7f74a406a750 2024-04-23T18:35:06.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.082+0000 7f74a27fc700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f748c0a3d50 0x7f748c0a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f749400b5c0 tx=0x7f7494005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:06.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.089+0000 7f748bfff700 1 -- 172.21.15.132:0/3669581246 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7498020020 con 0x7f74a406a750 2024-04-23T18:35:06.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.113+0000 7f0647c9f700 1 -- 172.21.15.132:0/1680919036 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0640109890 con 0x7f0640107990 2024-04-23T18:35:06.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.113+0000 7f0636ffd700 1 -- 172.21.15.132:0/1680919036 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f063809e970 con 0x7f0640107990 2024-04-23T18:35:06.105 INFO:teuthology.orchestra.run.smithi132.stdout:55834574924 2024-04-23T18:35:06.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.116+0000 7f0647c9f700 1 -- 172.21.15.132:0/1680919036 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f062c0a3d50 msgr2=0x7f062c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:06.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.116+0000 7f0647c9f700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f062c0a3d50 0x7f062c0a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f063c006fd0 tx=0x7f063c008040 comp rx=0 tx=0).stop 2024-04-23T18:35:06.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.116+0000 7f0647c9f700 1 -- 172.21.15.132:0/1680919036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640107990 msgr2=0x7f06401a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:06.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.116+0000 7f0647c9f700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640107990 0x7f06401a4f80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f06380062a0 tx=0x7f0638004ed0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.117+0000 7f0647c9f700 1 -- 172.21.15.132:0/1680919036 shutdown_connections 2024-04-23T18:35:06.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.117+0000 7f0647c9f700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f062c0a3d50 0x7f062c0a6210 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.117+0000 7f0647c9f700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640107990 0x7f06401a4f80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.117+0000 7f0647c9f700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0640101940 0x7f0640199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.117+0000 7f0647c9f700 1 --2- 172.21.15.132:0/1680919036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0640102300 0x7f064019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.117+0000 7f0647c9f700 1 -- 172.21.15.132:0/1680919036 >> 172.21.15.132:0/1680919036 conn(0x7f06400788f0 msgr2=0x7f06400ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:06.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.118+0000 7f0647c9f700 1 -- 172.21.15.132:0/1680919036 shutdown_connections 2024-04-23T18:35:06.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.118+0000 7f0647c9f700 1 -- 172.21.15.132:0/1680919036 wait complete. 2024-04-23T18:35:06.112 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:06.112 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:06.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.305+0000 7f74a9539700 1 -- 172.21.15.132:0/3669581246 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f74900058f0 con 0x7f74a406a750 2024-04-23T18:35:06.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.306+0000 7f748bfff700 1 -- 172.21.15.132:0/3669581246 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f749801a3b0 con 0x7f74a406a750 2024-04-23T18:35:06.298 INFO:teuthology.orchestra.run.smithi132.stdout:292057776165 2024-04-23T18:35:06.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.308+0000 7f74a9539700 1 -- 172.21.15.132:0/3669581246 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f748c0a3d50 msgr2=0x7f748c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:06.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.308+0000 7f74a9539700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f748c0a3d50 0x7f748c0a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f749400b5c0 tx=0x7f7494005960 comp rx=0 tx=0).stop 2024-04-23T18:35:06.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.308+0000 7f74a9539700 1 -- 172.21.15.132:0/3669581246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a406a750 msgr2=0x7f74a4101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:06.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.308+0000 7f74a9539700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a406a750 0x7f74a4101a10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f749800d980 tx=0x7f749800dc90 comp rx=0 tx=0).stop 2024-04-23T18:35:06.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.308+0000 7f74a9539700 1 -- 172.21.15.132:0/3669581246 shutdown_connections 2024-04-23T18:35:06.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.308+0000 7f74a9539700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a4108a30 0x7f74a4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.308+0000 7f74a9539700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f748c0a3d50 0x7f748c0a6210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.308+0000 7f74a9539700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74a406a750 0x7f74a4101a10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.308+0000 7f74a9539700 1 --2- 172.21.15.132:0/3669581246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a406b0f0 0x7f74a4101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.308+0000 7f74a9539700 1 -- 172.21.15.132:0/3669581246 >> 172.21.15.132:0/3669581246 conn(0x7f74a40788f0 msgr2=0x7f74a40ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:06.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.308+0000 7f74a9539700 1 -- 172.21.15.132:0/3669581246 shutdown_connections 2024-04-23T18:35:06.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.308+0000 7f74a9539700 1 -- 172.21.15.132:0/3669581246 wait complete. 2024-04-23T18:35:06.327 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:06 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1680919036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T18:35:06.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:06 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1680919036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T18:35:06.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:06 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1680919036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T18:35:06.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.436+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/900957607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e88108a50 msgr2=0x7f9e88108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:06.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.436+0000 7f9e8fbb7700 1 --2- 172.21.15.132:0/900957607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e88108a50 0x7f9e88108e30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009990 tx=0x7f9e84009ca0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.436+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/900957607 shutdown_connections 2024-04-23T18:35:06.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.436+0000 7f9e8fbb7700 1 --2- 172.21.15.132:0/900957607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e88103450 0x7f9e8810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.436+0000 7f9e8fbb7700 1 --2- 172.21.15.132:0/900957607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e88108a50 0x7f9e88108e30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.436+0000 7f9e8fbb7700 1 --2- 172.21.15.132:0/900957607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88102a90 0x7f9e88102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.436+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/900957607 >> 172.21.15.132:0/900957607 conn(0x7f9e88079ba0 msgr2=0x7f9e88079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.436+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/900957607 shutdown_connections 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.436+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/900957607 wait complete. 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8fbb7700 1 Processor -- start 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8fbb7700 1 -- start start 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8fbb7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88102a90 0x7f9e88199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8fbb7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e88103450 0x7f9e88199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8fbb7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e88108a50 0x7f9e8819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8fbb7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88112c00 con 0x7f9e88108a50 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8fbb7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88112a80 con 0x7f9e88103450 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8fbb7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88112d80 con 0x7f9e88102a90 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8e154700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e88108a50 0x7f9e8819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:06.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8e154700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e88108a50 0x7f9e8819da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54296/0 (socket says 172.21.15.132:54296) 2024-04-23T18:35:06.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8e154700 1 -- 172.21.15.132:0/2979996904 learned_addr learned my addr 172.21.15.132:0/2979996904 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:06.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8d953700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88102a90 0x7f9e88199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:06.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8d152700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e88103450 0x7f9e88199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:06.859 INFO:teuthology.orchestra.run.smithi132.stdout:34359738450 2024-04-23T18:35:06.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8e154700 1 -- 172.21.15.132:0/2979996904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88102a90 msgr2=0x7f9e88199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:06.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8e154700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88102a90 0x7f9e88199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8e154700 1 -- 172.21.15.132:0/2979996904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e88103450 msgr2=0x7f9e88199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:06.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8e154700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e88103450 0x7f9e88199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8e154700 1 -- 172.21.15.132:0/2979996904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8819e140 con 0x7f9e88108a50 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.437+0000 7f9e8d953700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88102a90 0x7f9e88199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.438+0000 7f9e8e154700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e88108a50 0x7f9e8819da00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000ada0 tx=0x7f9e8000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.438+0000 7f9e7effd700 1 -- 172.21.15.132:0/2979996904 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80017020 con 0x7f9e88108a50 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.438+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/2979996904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8819e430 con 0x7f9e88108a50 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.438+0000 7f9e7effd700 1 -- 172.21.15.132:0/2979996904 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e8000ce30 con 0x7f9e88108a50 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.438+0000 7f9e7effd700 1 -- 172.21.15.132:0/2979996904 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e800054a0 con 0x7f9e88108a50 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.438+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/2979996904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e881a5cb0 con 0x7f9e88108a50 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.439+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/2979996904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e88069730 con 0x7f9e88108a50 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.440+0000 7f9e7effd700 1 -- 172.21.15.132:0/2979996904 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9e8000c980 con 0x7f9e88108a50 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.440+0000 7f9e7effd700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9e740a3cd0 0x7f9e740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.441+0000 7f9e8d953700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9e740a3cd0 0x7f9e740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.441+0000 7f9e7effd700 1 -- 172.21.15.132:0/2979996904 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9e800d0320 con 0x7f9e88108a50 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.441+0000 7f9e8d953700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9e740a3cd0 0x7f9e740a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e84006010 tx=0x7f9e8400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.444+0000 7f9e7effd700 1 -- 172.21.15.132:0/2979996904 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e80021020 con 0x7f9e88108a50 2024-04-23T18:35:06.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.647+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/2979996904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9e8802df80 con 0x7f9e88108a50 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.647+0000 7f9e7effd700 1 -- 172.21.15.132:0/2979996904 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9e8009cc20 con 0x7f9e88108a50 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.649+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/2979996904 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9e740a3cd0 msgr2=0x7f9e740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.649+0000 7f9e8fbb7700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9e740a3cd0 0x7f9e740a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e84006010 tx=0x7f9e8400b540 comp rx=0 tx=0).stop 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.649+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/2979996904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e88108a50 msgr2=0x7f9e8819da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.649+0000 7f9e8fbb7700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e88108a50 0x7f9e8819da00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000ada0 tx=0x7f9e8000c330 comp rx=0 tx=0).stop 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.649+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/2979996904 shutdown_connections 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.649+0000 7f9e8fbb7700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9e740a3cd0 0x7f9e740a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.649+0000 7f9e8fbb7700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e88108a50 0x7f9e8819da00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.649+0000 7f9e8fbb7700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e88102a90 0x7f9e88199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.649+0000 7f9e8fbb7700 1 --2- 172.21.15.132:0/2979996904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e88103450 0x7f9e88199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.649+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/2979996904 >> 172.21.15.132:0/2979996904 conn(0x7f9e88079ba0 msgr2=0x7f9e88101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.649+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/2979996904 shutdown_connections 2024-04-23T18:35:06.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:06.650+0000 7f9e8fbb7700 1 -- 172.21.15.132:0/2979996904 wait complete. 2024-04-23T18:35:07.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:07 smithi132 ceph-mon[29178]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:35:07.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:07 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3669581246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T18:35:07.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:07 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2979996904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T18:35:07.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:07 smithi148 ceph-mon[32311]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:35:07.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:07 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3669581246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T18:35:07.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:07 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2979996904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T18:35:07.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:07 smithi157 ceph-mon[32580]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:35:07.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:07 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3669581246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T18:35:07.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:07 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2979996904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T18:35:07.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.816+0000 7f2eee486700 1 -- 172.21.15.132:0/2419383217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ee8108a20 msgr2=0x7f2ee8108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:07.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.816+0000 7f2eee486700 1 --2- 172.21.15.132:0/2419383217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ee8108a20 0x7f2ee8108e00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009990 tx=0x7f2ed8009ca0 comp rx=0 tx=0).stop 2024-04-23T18:35:07.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.816+0000 7f2eee486700 1 -- 172.21.15.132:0/2419383217 shutdown_connections 2024-04-23T18:35:07.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.816+0000 7f2eee486700 1 --2- 172.21.15.132:0/2419383217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee806b980 0x7f2ee810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:07.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.816+0000 7f2eee486700 1 --2- 172.21.15.132:0/2419383217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ee8108a20 0x7f2ee8108e00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:07.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.816+0000 7f2eee486700 1 --2- 172.21.15.132:0/2419383217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee806afe0 0x7f2ee806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:07.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.816+0000 7f2eee486700 1 -- 172.21.15.132:0/2419383217 >> 172.21.15.132:0/2419383217 conn(0x7f2ee80788b0 msgr2=0x7f2ee8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:07.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.816+0000 7f2eee486700 1 -- 172.21.15.132:0/2419383217 shutdown_connections 2024-04-23T18:35:07.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.816+0000 7f2eee486700 1 -- 172.21.15.132:0/2419383217 wait complete. 2024-04-23T18:35:07.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.816+0000 7f2eee486700 1 Processor -- start 2024-04-23T18:35:07.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eee486700 1 -- start start 2024-04-23T18:35:07.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eee486700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee806afe0 0x7f2ee81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:07.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eee486700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee806b980 0x7f2ee8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:07.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eee486700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ee8108a20 0x7f2ee819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:07.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eee486700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee8112ca0 con 0x7f2ee8108a20 2024-04-23T18:35:07.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eee486700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee8112b20 con 0x7f2ee806afe0 2024-04-23T18:35:07.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eee486700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee8112e20 con 0x7f2ee806b980 2024-04-23T18:35:07.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eeca23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ee8108a20 0x7f2ee819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:07.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eeca23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ee8108a20 0x7f2ee819db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54306/0 (socket says 172.21.15.132:54306) 2024-04-23T18:35:07.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eeca23700 1 -- 172.21.15.132:0/3803082226 learned_addr learned my addr 172.21.15.132:0/3803082226 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:07.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2ee77fe700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee806b980 0x7f2ee8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:07.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2ee7fff700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee806afe0 0x7f2ee81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:07.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eeca23700 1 -- 172.21.15.132:0/3803082226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee806b980 msgr2=0x7f2ee8199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:07.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eeca23700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee806b980 0x7f2ee8199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:07.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eeca23700 1 -- 172.21.15.132:0/3803082226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee806afe0 msgr2=0x7f2ee81992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:07.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eeca23700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee806afe0 0x7f2ee81992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:07.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.817+0000 7f2eeca23700 1 -- 172.21.15.132:0/3803082226 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee819e2d0 con 0x7f2ee8108a20 2024-04-23T18:35:07.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.818+0000 7f2eeca23700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ee8108a20 0x7f2ee819db90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000b970 tx=0x7f2ee000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:07.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.818+0000 7f2ee57fa700 1 -- 172.21.15.132:0/3803082226 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0019070 con 0x7f2ee8108a20 2024-04-23T18:35:07.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.818+0000 7f2eee486700 1 -- 172.21.15.132:0/3803082226 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee819e5c0 con 0x7f2ee8108a20 2024-04-23T18:35:07.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.818+0000 7f2ee57fa700 1 -- 172.21.15.132:0/3803082226 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ee0014440 con 0x7f2ee8108a20 2024-04-23T18:35:07.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.818+0000 7f2ee57fa700 1 -- 172.21.15.132:0/3803082226 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000ce60 con 0x7f2ee8108a20 2024-04-23T18:35:07.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.818+0000 7f2eee486700 1 -- 172.21.15.132:0/3803082226 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee81a5e40 con 0x7f2ee8108a20 2024-04-23T18:35:07.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.819+0000 7f2eee486700 1 -- 172.21.15.132:0/3803082226 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee8069730 con 0x7f2ee8108a20 2024-04-23T18:35:07.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.820+0000 7f2ee57fa700 1 -- 172.21.15.132:0/3803082226 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2ee001c070 con 0x7f2ee8108a20 2024-04-23T18:35:07.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.820+0000 7f2ee57fa700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2ed00a3cd0 0x7f2ed00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:07.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.820+0000 7f2ee7fff700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2ed00a3cd0 0x7f2ed00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:07.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.820+0000 7f2ee57fa700 1 -- 172.21.15.132:0/3803082226 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f2ee00cf860 con 0x7f2ee8108a20 2024-04-23T18:35:07.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.821+0000 7f2ee7fff700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2ed00a3cd0 0x7f2ed00a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8006010 tx=0x7f2ed801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:07.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:07.825+0000 7f2ee57fa700 1 -- 172.21.15.132:0/3803082226 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ee009c1e0 con 0x7f2ee8108a20 2024-04-23T18:35:08.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.029+0000 7f2eee486700 1 -- 172.21.15.132:0/3803082226 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f2ee819a620 con 0x7f2ee8108a20 2024-04-23T18:35:08.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.030+0000 7f2ee57fa700 1 -- 172.21.15.132:0/3803082226 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2ee009bae0 con 0x7f2ee8108a20 2024-04-23T18:35:08.022 INFO:teuthology.orchestra.run.smithi132.stdout:313532612640 2024-04-23T18:35:08.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.032+0000 7f2eee486700 1 -- 172.21.15.132:0/3803082226 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2ed00a3cd0 msgr2=0x7f2ed00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:08.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.032+0000 7f2eee486700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2ed00a3cd0 0x7f2ed00a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8006010 tx=0x7f2ed801a040 comp rx=0 tx=0).stop 2024-04-23T18:35:08.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.032+0000 7f2eee486700 1 -- 172.21.15.132:0/3803082226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ee8108a20 msgr2=0x7f2ee819db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:08.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.032+0000 7f2eee486700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ee8108a20 0x7f2ee819db90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000b970 tx=0x7f2ee000bd30 comp rx=0 tx=0).stop 2024-04-23T18:35:08.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.032+0000 7f2eee486700 1 -- 172.21.15.132:0/3803082226 shutdown_connections 2024-04-23T18:35:08.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.032+0000 7f2eee486700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f2ed00a3cd0 0x7f2ed00a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.032+0000 7f2eee486700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ee8108a20 0x7f2ee819db90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.032+0000 7f2eee486700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee806afe0 0x7f2ee81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.032+0000 7f2eee486700 1 --2- 172.21.15.132:0/3803082226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee806b980 0x7f2ee8199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.032+0000 7f2eee486700 1 -- 172.21.15.132:0/3803082226 >> 172.21.15.132:0/3803082226 conn(0x7f2ee80788b0 msgr2=0x7f2ee80ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:08.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.032+0000 7f2eee486700 1 -- 172.21.15.132:0/3803082226 shutdown_connections 2024-04-23T18:35:08.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:08.032+0000 7f2eee486700 1 -- 172.21.15.132:0/3803082226 wait complete. 2024-04-23T18:35:08.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:08 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3803082226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T18:35:08.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:08 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3803082226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T18:35:08.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:08 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3803082226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T18:35:08.718 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612635 got 313532612640 for osd.10 2024-04-23T18:35:08.718 DEBUG:teuthology.parallel:result is None 2024-04-23T18:35:09.461 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:09 smithi132 ceph-mon[29178]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:35:09.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:09 smithi148 ceph-mon[32311]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:35:09.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:09 smithi157 ceph-mon[32580]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:35:09.893 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738450 for osd.0 2024-04-23T18:35:09.893 DEBUG:teuthology.parallel:result is None 2024-04-23T18:35:10.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.110+0000 7f91bf811700 1 -- 172.21.15.132:0/2559724672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91b8108a70 msgr2=0x7f91b8108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:10.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.110+0000 7f91bf811700 1 --2- 172.21.15.132:0/2559724672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91b8108a70 0x7f91b8108e50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009990 tx=0x7f91b4009ca0 comp rx=0 tx=0).stop 2024-04-23T18:35:10.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.110+0000 7f91bf811700 1 -- 172.21.15.132:0/2559724672 shutdown_connections 2024-04-23T18:35:10.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.110+0000 7f91bf811700 1 --2- 172.21.15.132:0/2559724672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91b8103480 0x7f91b810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:10.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.110+0000 7f91bf811700 1 --2- 172.21.15.132:0/2559724672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91b8108a70 0x7f91b8108e50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:10.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.110+0000 7f91bf811700 1 --2- 172.21.15.132:0/2559724672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91b8102ac0 0x7f91b8102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:10.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.110+0000 7f91bf811700 1 -- 172.21.15.132:0/2559724672 >> 172.21.15.132:0/2559724672 conn(0x7f91b8079bb0 msgr2=0x7f91b8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:10.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.110+0000 7f91bf811700 1 -- 172.21.15.132:0/2559724672 shutdown_connections 2024-04-23T18:35:10.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.110+0000 7f91bf811700 1 -- 172.21.15.132:0/2559724672 wait complete. 2024-04-23T18:35:10.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.110+0000 7f91bf811700 1 Processor -- start 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.110+0000 7f91bf811700 1 -- start start 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bf811700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91b8102ac0 0x7f91b8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bf811700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91b8103480 0x7f91b8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bf811700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91b8108a70 0x7f91b819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bf811700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8112d00 con 0x7f91b8103480 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bf811700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8112b80 con 0x7f91b8102ac0 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bf811700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8112e80 con 0x7f91b8108a70 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bcdac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91b8103480 0x7f91b8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bcdac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91b8103480 0x7f91b8199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35174/0 (socket says 172.21.15.132:35174) 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bcdac700 1 -- 172.21.15.132:0/4163243318 learned_addr learned my addr 172.21.15.132:0/4163243318 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bd5ad700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91b8102ac0 0x7f91b8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bddae700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91b8108a70 0x7f91b819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bcdac700 1 -- 172.21.15.132:0/4163243318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91b8108a70 msgr2=0x7f91b819db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:10.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bcdac700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91b8108a70 0x7f91b819db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:10.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bcdac700 1 -- 172.21.15.132:0/4163243318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91b8102ac0 msgr2=0x7f91b8199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:10.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bcdac700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91b8102ac0 0x7f91b8199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:10.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bcdac700 1 -- 172.21.15.132:0/4163243318 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b819e240 con 0x7f91b8103480 2024-04-23T18:35:10.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bddae700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91b8108a70 0x7f91b819db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:10.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bd5ad700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91b8102ac0 0x7f91b8199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:10.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.111+0000 7f91bcdac700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91b8103480 0x7f91b8199770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91a800b530 tx=0x7f91a800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:10.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.112+0000 7f91ae7fc700 1 -- 172.21.15.132:0/4163243318 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8015020 con 0x7f91b8103480 2024-04-23T18:35:10.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.112+0000 7f91bf811700 1 -- 172.21.15.132:0/4163243318 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b819e530 con 0x7f91b8103480 2024-04-23T18:35:10.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.112+0000 7f91ae7fc700 1 -- 172.21.15.132:0/4163243318 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a8011ce0 con 0x7f91b8103480 2024-04-23T18:35:10.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.112+0000 7f91ae7fc700 1 -- 172.21.15.132:0/4163243318 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8010d20 con 0x7f91b8103480 2024-04-23T18:35:10.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.112+0000 7f91bf811700 1 -- 172.21.15.132:0/4163243318 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b81a5db0 con 0x7f91b8103480 2024-04-23T18:35:10.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.113+0000 7f91bf811700 1 -- 172.21.15.132:0/4163243318 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b8069730 con 0x7f91b8103480 2024-04-23T18:35:10.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.115+0000 7f91ae7fc700 1 -- 172.21.15.132:0/4163243318 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f91a8018020 con 0x7f91b8103480 2024-04-23T18:35:10.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.115+0000 7f91ae7fc700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f91a40a3cd0 0x7f91a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:10.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.115+0000 7f91bd5ad700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f91a40a3cd0 0x7f91a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:10.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.115+0000 7f91ae7fc700 1 -- 172.21.15.132:0/4163243318 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f91a80d0210 con 0x7f91b8103480 2024-04-23T18:35:10.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.116+0000 7f91bd5ad700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f91a40a3cd0 0x7f91a40a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91b4006010 tx=0x7f91b400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:10.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.119+0000 7f91ae7fc700 1 -- 172.21.15.132:0/4163243318 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91a801f020 con 0x7f91b8103480 2024-04-23T18:35:10.317 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.324+0000 7f91bf811700 1 -- 172.21.15.132:0/4163243318 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f91b802df80 con 0x7f91b8103480 2024-04-23T18:35:10.317 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.325+0000 7f91ae7fc700 1 -- 172.21.15.132:0/4163243318 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f91a809cb90 con 0x7f91b8103480 2024-04-23T18:35:10.317 INFO:teuthology.orchestra.run.smithi132.stdout:107374182468 2024-04-23T18:35:10.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.327+0000 7f91bf811700 1 -- 172.21.15.132:0/4163243318 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f91a40a3cd0 msgr2=0x7f91a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:10.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.327+0000 7f91bf811700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f91a40a3cd0 0x7f91a40a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91b4006010 tx=0x7f91b400b540 comp rx=0 tx=0).stop 2024-04-23T18:35:10.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.327+0000 7f91bf811700 1 -- 172.21.15.132:0/4163243318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91b8103480 msgr2=0x7f91b8199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:10.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.327+0000 7f91bf811700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91b8103480 0x7f91b8199770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91a800b530 tx=0x7f91a800b8f0 comp rx=0 tx=0).stop 2024-04-23T18:35:10.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.327+0000 7f91bf811700 1 -- 172.21.15.132:0/4163243318 shutdown_connections 2024-04-23T18:35:10.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.327+0000 7f91bf811700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91b8108a70 0x7f91b819db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:10.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.327+0000 7f91bf811700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91b8102ac0 0x7f91b8199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:10.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.327+0000 7f91bf811700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f91a40a3cd0 0x7f91a40a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:10.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.327+0000 7f91bf811700 1 --2- 172.21.15.132:0/4163243318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91b8103480 0x7f91b8199770 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:10.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.327+0000 7f91bf811700 1 -- 172.21.15.132:0/4163243318 >> 172.21.15.132:0/4163243318 conn(0x7f91b8079bb0 msgr2=0x7f91b81019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:10.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.327+0000 7f91bf811700 1 -- 172.21.15.132:0/4163243318 shutdown_connections 2024-04-23T18:35:10.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:10.327+0000 7f91bf811700 1 -- 172.21.15.132:0/4163243318 wait complete. 2024-04-23T18:35:10.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776159 got 292057776165 for osd.9 2024-04-23T18:35:10.600 DEBUG:teuthology.parallel:result is None 2024-04-23T18:35:11.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.325+0000 7f460a995700 1 -- 172.21.15.132:0/1295980090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f460406afe0 msgr2=0x7f460406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:11.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.325+0000 7f460a995700 1 --2- 172.21.15.132:0/1295980090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f460406afe0 0x7f460406b440 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009990 tx=0x7f45f8009ca0 comp rx=0 tx=0).stop 2024-04-23T18:35:11.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.325+0000 7f460a995700 1 -- 172.21.15.132:0/1295980090 shutdown_connections 2024-04-23T18:35:11.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.325+0000 7f460a995700 1 --2- 172.21.15.132:0/1295980090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f460406b980 0x7f460410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:11.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.325+0000 7f460a995700 1 --2- 172.21.15.132:0/1295980090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4604108a20 0x7f4604108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:11.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.325+0000 7f460a995700 1 --2- 172.21.15.132:0/1295980090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f460406afe0 0x7f460406b440 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:11.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.325+0000 7f460a995700 1 -- 172.21.15.132:0/1295980090 >> 172.21.15.132:0/1295980090 conn(0x7f46040788b0 msgr2=0x7f4604078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:11.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.325+0000 7f460a995700 1 -- 172.21.15.132:0/1295980090 shutdown_connections 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.325+0000 7f460a995700 1 -- 172.21.15.132:0/1295980090 wait complete. 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.325+0000 7f460a995700 1 Processor -- start 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.325+0000 7f460a995700 1 -- start start 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f460a995700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f460406afe0 0x7f4604101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f460a995700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460406b980 0x7f4604101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f460a995700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4604108a20 0x7f4604102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f460a995700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4604112c20 con 0x7f4604108a20 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f460a995700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4604112aa0 con 0x7f460406afe0 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f460a995700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4604112da0 con 0x7f460406b980 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f4608f32700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4604108a20 0x7f4604102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f4608f32700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4604108a20 0x7f4604102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35182/0 (socket says 172.21.15.132:35182) 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f4608f32700 1 -- 172.21.15.132:0/1995369282 learned_addr learned my addr 172.21.15.132:0/1995369282 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f4603fff700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f460406afe0 0x7f4604101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:11.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f46037fe700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460406b980 0x7f4604101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f4608f32700 1 -- 172.21.15.132:0/1995369282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460406b980 msgr2=0x7f4604101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f4608f32700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460406b980 0x7f4604101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f4608f32700 1 -- 172.21.15.132:0/1995369282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f460406afe0 msgr2=0x7f4604101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f4608f32700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f460406afe0 0x7f4604101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f4608f32700 1 -- 172.21.15.132:0/1995369282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46041a76e0 con 0x7f4604108a20 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f46037fe700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460406b980 0x7f4604101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.326+0000 7f4608f32700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4604108a20 0x7f4604102480 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00ada0 tx=0x7f45fc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.327+0000 7f46017fa700 1 -- 172.21.15.132:0/1995369282 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc017020 con 0x7f4604108a20 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.327+0000 7f460a995700 1 -- 172.21.15.132:0/1995369282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46041a79d0 con 0x7f4604108a20 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.327+0000 7f46017fa700 1 -- 172.21.15.132:0/1995369282 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45fc00ce30 con 0x7f4604108a20 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.327+0000 7f46017fa700 1 -- 172.21.15.132:0/1995369282 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc005350 con 0x7f4604108a20 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.327+0000 7f460a995700 1 -- 172.21.15.132:0/1995369282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46041a7ec0 con 0x7f4604108a20 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.328+0000 7f460a995700 1 -- 172.21.15.132:0/1995369282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f460410b020 con 0x7f4604108a20 2024-04-23T18:35:11.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.329+0000 7f46017fa700 1 -- 172.21.15.132:0/1995369282 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f45fc00c980 con 0x7f4604108a20 2024-04-23T18:35:11.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.330+0000 7f46017fa700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45ec0a3cd0 0x7f45ec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:11.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.330+0000 7f4603fff700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45ec0a3cd0 0x7f45ec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:11.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.330+0000 7f46017fa700 1 -- 172.21.15.132:0/1995369282 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f45fc0d0280 con 0x7f4604108a20 2024-04-23T18:35:11.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.330+0000 7f4603fff700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45ec0a3cd0 0x7f45ec0a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009a10 tx=0x7f45f4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:11.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.334+0000 7f46017fa700 1 -- 172.21.15.132:0/1995369282 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45fc021020 con 0x7f4604108a20 2024-04-23T18:35:11.412 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:11 smithi132 ceph-mon[29178]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-23T18:35:11.412 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:11 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/4163243318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T18:35:11.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:11 smithi148 ceph-mon[32311]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-23T18:35:11.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:11 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/4163243318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T18:35:11.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.538+0000 7f460a995700 1 -- 172.21.15.132:0/1995369282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f460402df80 con 0x7f4604108a20 2024-04-23T18:35:11.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.539+0000 7f46017fa700 1 -- 172.21.15.132:0/1995369282 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f45fc01a310 con 0x7f4604108a20 2024-04-23T18:35:11.531 INFO:teuthology.orchestra.run.smithi132.stdout:163208757305 2024-04-23T18:35:11.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.541+0000 7f460a995700 1 -- 172.21.15.132:0/1995369282 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45ec0a3cd0 msgr2=0x7f45ec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:11.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.541+0000 7f460a995700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45ec0a3cd0 0x7f45ec0a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009a10 tx=0x7f45f4009190 comp rx=0 tx=0).stop 2024-04-23T18:35:11.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.541+0000 7f460a995700 1 -- 172.21.15.132:0/1995369282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4604108a20 msgr2=0x7f4604102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:11.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.541+0000 7f460a995700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4604108a20 0x7f4604102480 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00ada0 tx=0x7f45fc00c330 comp rx=0 tx=0).stop 2024-04-23T18:35:11.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.541+0000 7f460a995700 1 -- 172.21.15.132:0/1995369282 shutdown_connections 2024-04-23T18:35:11.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.541+0000 7f460a995700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f45ec0a3cd0 0x7f45ec0a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:11.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.541+0000 7f460a995700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4604108a20 0x7f4604102480 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:11.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.541+0000 7f460a995700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f460406afe0 0x7f4604101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:11.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.541+0000 7f460a995700 1 --2- 172.21.15.132:0/1995369282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460406b980 0x7f4604101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:11.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.541+0000 7f460a995700 1 -- 172.21.15.132:0/1995369282 >> 172.21.15.132:0/1995369282 conn(0x7f46040788b0 msgr2=0x7f46040ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:11.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.541+0000 7f460a995700 1 -- 172.21.15.132:0/1995369282 shutdown_connections 2024-04-23T18:35:11.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:11.541+0000 7f460a995700 1 -- 172.21.15.132:0/1995369282 wait complete. 2024-04-23T18:35:11.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:11 smithi157 ceph-mon[32580]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-23T18:35:11.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:11 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/4163243318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T18:35:12.341 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:12 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1995369282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T18:35:12.368 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182468 for osd.3 2024-04-23T18:35:12.368 DEBUG:teuthology.parallel:result is None 2024-04-23T18:35:12.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:12 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1995369282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T18:35:12.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:12 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1995369282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T18:35:12.746 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574924 for osd.1 2024-04-23T18:35:12.746 DEBUG:teuthology.parallel:result is None 2024-04-23T18:35:13.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:13 smithi148 ceph-mon[32311]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-23T18:35:13.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:13 smithi157 ceph-mon[32580]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-23T18:35:13.592 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:13 smithi132 ceph-mon[29178]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-23T18:35:14.048 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757299 got 163208757305 for osd.5 2024-04-23T18:35:14.048 DEBUG:teuthology.parallel:result is None 2024-04-23T18:35:14.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.886+0000 7f8f218d8700 1 -- 172.21.15.132:0/1657819535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f1c1019d0 msgr2=0x7f8f1c10f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:14.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.886+0000 7f8f218d8700 1 --2- 172.21.15.132:0/1657819535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f1c1019d0 0x7f8f1c10f910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009990 tx=0x7f8f14009ca0 comp rx=0 tx=0).stop 2024-04-23T18:35:14.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.886+0000 7f8f218d8700 1 -- 172.21.15.132:0/1657819535 shutdown_connections 2024-04-23T18:35:14.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.886+0000 7f8f218d8700 1 --2- 172.21.15.132:0/1657819535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f1c1019d0 0x7f8f1c10f910 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:14.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.886+0000 7f8f218d8700 1 --2- 172.21.15.132:0/1657819535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c108ad0 0x7f8f1c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:14.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.886+0000 7f8f218d8700 1 --2- 172.21.15.132:0/1657819535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1c101030 0x7f8f1c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:14.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.886+0000 7f8f218d8700 1 -- 172.21.15.132:0/1657819535 >> 172.21.15.132:0/1657819535 conn(0x7f8f1c077fb0 msgr2=0x7f8f1c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:14.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.886+0000 7f8f218d8700 1 -- 172.21.15.132:0/1657819535 shutdown_connections 2024-04-23T18:35:14.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.886+0000 7f8f218d8700 1 -- 172.21.15.132:0/1657819535 wait complete. 2024-04-23T18:35:14.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.887+0000 7f8f218d8700 1 Processor -- start 2024-04-23T18:35:14.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.887+0000 7f8f218d8700 1 -- start start 2024-04-23T18:35:14.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.887+0000 7f8f218d8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1c101030 0x7f8f1c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.887+0000 7f8f218d8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c1019d0 0x7f8f1c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.887+0000 7f8f218d8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f1c108ad0 0x7f8f1c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.887+0000 7f8f218d8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c112c40 con 0x7f8f1c108ad0 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.887+0000 7f8f218d8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c112ac0 con 0x7f8f1c101030 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.887+0000 7f8f218d8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c112dc0 con 0x7f8f1c1019d0 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.887+0000 7f8f1b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f1c108ad0 0x7f8f1c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.887+0000 7f8f1b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f1c108ad0 0x7f8f1c19d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35204/0 (socket says 172.21.15.132:35204) 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.887+0000 7f8f1b7fe700 1 -- 172.21.15.132:0/1279165620 learned_addr learned my addr 172.21.15.132:0/1279165620 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.887+0000 7f8f1affd700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1c101030 0x7f8f1c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f1a7fc700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c1019d0 0x7f8f1c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f1b7fe700 1 -- 172.21.15.132:0/1279165620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c1019d0 msgr2=0x7f8f1c199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f1b7fe700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c1019d0 0x7f8f1c199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f1b7fe700 1 -- 172.21.15.132:0/1279165620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1c101030 msgr2=0x7f8f1c199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:14.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f1b7fe700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1c101030 0x7f8f1c199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f1b7fe700 1 -- 172.21.15.132:0/1279165620 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c19e120 con 0x7f8f1c108ad0 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f1affd700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1c101030 0x7f8f1c199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f1b7fe700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f1c108ad0 0x7f8f1c19d9e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400c010 tx=0x7f8f14009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f208d6700 1 -- 172.21.15.132:0/1279165620 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14027070 con 0x7f8f1c108ad0 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f208d6700 1 -- 172.21.15.132:0/1279165620 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f14003fa0 con 0x7f8f1c108ad0 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f218d8700 1 -- 172.21.15.132:0/1279165620 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1c19e3b0 con 0x7f8f1c108ad0 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f208d6700 1 -- 172.21.15.132:0/1279165620 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14022d70 con 0x7f8f1c108ad0 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.888+0000 7f8f218d8700 1 -- 172.21.15.132:0/1279165620 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1c1a5c90 con 0x7f8f1c108ad0 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.889+0000 7f8f218d8700 1 -- 172.21.15.132:0/1279165620 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1c104350 con 0x7f8f1c108ad0 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.890+0000 7f8f208d6700 1 -- 172.21.15.132:0/1279165620 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8f1402a070 con 0x7f8f1c108ad0 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.890+0000 7f8f208d6700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8f040a3cd0 0x7f8f040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.891+0000 7f8f1affd700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8f040a3cd0 0x7f8f040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.891+0000 7f8f208d6700 1 -- 172.21.15.132:0/1279165620 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f8f140d1d30 con 0x7f8f1c108ad0 2024-04-23T18:35:14.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.891+0000 7f8f1affd700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8f040a3cd0 0x7f8f040a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009fa0 tx=0x7f8f0c009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:14.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:14.894+0000 7f8f208d6700 1 -- 172.21.15.132:0/1279165620 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f14031020 con 0x7f8f1c108ad0 2024-04-23T18:35:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.096+0000 7fb28ff24700 1 -- 172.21.15.132:0/630125732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28806b980 msgr2=0x7fb28810f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.096+0000 7fb28ff24700 1 --2- 172.21.15.132:0/630125732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28806b980 0x7fb28810f8f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb28000b0a0 tx=0x7fb28000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.096+0000 7fb28ff24700 1 -- 172.21.15.132:0/630125732 shutdown_connections 2024-04-23T18:35:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.096+0000 7fb28ff24700 1 --2- 172.21.15.132:0/630125732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28806b980 0x7fb28810f8f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 --2- 172.21.15.132:0/630125732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb288108a20 0x7fb288108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 --2- 172.21.15.132:0/630125732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb28806afe0 0x7fb28806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 -- 172.21.15.132:0/630125732 >> 172.21.15.132:0/630125732 conn(0x7fb2880788b0 msgr2=0x7fb288078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 -- 172.21.15.132:0/630125732 shutdown_connections 2024-04-23T18:35:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 -- 172.21.15.132:0/630125732 wait complete. 2024-04-23T18:35:15.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 Processor -- start 2024-04-23T18:35:15.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 -- start start 2024-04-23T18:35:15.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb28806afe0 0x7fb2881990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:15.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28806b980 0x7fb288199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:15.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb288108a20 0x7fb28819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:15.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb288112ba0 con 0x7fb28806b980 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb288112a20 con 0x7fb28806afe0 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.097+0000 7fb28ff24700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb288112d20 con 0x7fb288108a20 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb28d4bf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28806b980 0x7fb288199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb28d4bf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28806b980 0x7fb288199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35228/0 (socket says 172.21.15.132:35228) 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb28d4bf700 1 -- 172.21.15.132:0/2518497206 learned_addr learned my addr 172.21.15.132:0/2518497206 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb28dcc0700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb28806afe0 0x7fb2881990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb28d4bf700 1 -- 172.21.15.132:0/2518497206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb288108a20 msgr2=0x7fb28819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb28d4bf700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb288108a20 0x7fb28819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb28d4bf700 1 -- 172.21.15.132:0/2518497206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb28806afe0 msgr2=0x7fb2881990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb28d4bf700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb28806afe0 0x7fb2881990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb28d4bf700 1 -- 172.21.15.132:0/2518497206 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28819e090 con 0x7fb28806b980 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb28d4bf700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28806b980 0x7fb288199610 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb27800e9a0 tx=0x7fb27800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb27effd700 1 -- 172.21.15.132:0/2518497206 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27800cbd0 con 0x7fb28806b980 2024-04-23T18:35:15.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb28ff24700 1 -- 172.21.15.132:0/2518497206 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28819e380 con 0x7fb28806b980 2024-04-23T18:35:15.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb27effd700 1 -- 172.21.15.132:0/2518497206 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb27800cd70 con 0x7fb28806b980 2024-04-23T18:35:15.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb27effd700 1 -- 172.21.15.132:0/2518497206 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278018680 con 0x7fb28806b980 2024-04-23T18:35:15.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.098+0000 7fb28ff24700 1 -- 172.21.15.132:0/2518497206 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2881a5c00 con 0x7fb28806b980 2024-04-23T18:35:15.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.099+0000 7fb28ff24700 1 -- 172.21.15.132:0/2518497206 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb288069730 con 0x7fb28806b980 2024-04-23T18:35:15.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.101+0000 7fb27effd700 1 -- 172.21.15.132:0/2518497206 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb278010970 con 0x7fb28806b980 2024-04-23T18:35:15.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.102+0000 7fb27effd700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb2740a3cd0 0x7fb2740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:15.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.102+0000 7fb28dcc0700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb2740a3cd0 0x7fb2740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:15.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.102+0000 7fb27effd700 1 -- 172.21.15.132:0/2518497206 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb2780d08a0 con 0x7fb28806b980 2024-04-23T18:35:15.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.102+0000 7fb28dcc0700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb2740a3cd0 0x7fb2740a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb284005950 tx=0x7fb2840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.107+0000 7f8f218d8700 1 -- 172.21.15.132:0/1279165620 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8f1c19a470 con 0x7f8f1c108ad0 2024-04-23T18:35:15.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.107+0000 7fb27effd700 1 -- 172.21.15.132:0/2518497206 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb27809d110 con 0x7fb28806b980 2024-04-23T18:35:15.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.107+0000 7f8f208d6700 1 -- 172.21.15.132:0/1279165620 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8f14013070 con 0x7f8f1c108ad0 2024-04-23T18:35:15.099 INFO:teuthology.orchestra.run.smithi132.stdout:141733920832 2024-04-23T18:35:15.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.109+0000 7f8f218d8700 1 -- 172.21.15.132:0/1279165620 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8f040a3cd0 msgr2=0x7f8f040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:15.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.109+0000 7f8f218d8700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8f040a3cd0 0x7f8f040a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009fa0 tx=0x7f8f0c009570 comp rx=0 tx=0).stop 2024-04-23T18:35:15.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.109+0000 7f8f218d8700 1 -- 172.21.15.132:0/1279165620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f1c108ad0 msgr2=0x7f8f1c19d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:15.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.109+0000 7f8f218d8700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f1c108ad0 0x7f8f1c19d9e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400c010 tx=0x7f8f14009ee0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.110+0000 7f8f218d8700 1 -- 172.21.15.132:0/1279165620 shutdown_connections 2024-04-23T18:35:15.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.110+0000 7f8f218d8700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8f040a3cd0 0x7f8f040a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.110+0000 7f8f218d8700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f1c108ad0 0x7f8f1c19d9e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.110+0000 7f8f218d8700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1c101030 0x7f8f1c199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.110+0000 7f8f218d8700 1 --2- 172.21.15.132:0/1279165620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1c1019d0 0x7f8f1c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.110+0000 7f8f218d8700 1 -- 172.21.15.132:0/1279165620 >> 172.21.15.132:0/1279165620 conn(0x7f8f1c077fb0 msgr2=0x7f8f1c0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.110+0000 7f8f218d8700 1 -- 172.21.15.132:0/1279165620 shutdown_connections 2024-04-23T18:35:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.110+0000 7f8f218d8700 1 -- 172.21.15.132:0/1279165620 wait complete. 2024-04-23T18:35:15.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.315+0000 7fb28ff24700 1 -- 172.21.15.132:0/2518497206 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb28802e140 con 0x7fb28806b980 2024-04-23T18:35:15.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.316+0000 7fb27effd700 1 -- 172.21.15.132:0/2518497206 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb27809ca10 con 0x7fb28806b980 2024-04-23T18:35:15.308 INFO:teuthology.orchestra.run.smithi132.stdout:257698037804 2024-04-23T18:35:15.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.318+0000 7fb28ff24700 1 -- 172.21.15.132:0/2518497206 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb2740a3cd0 msgr2=0x7fb2740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:15.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.318+0000 7fb28ff24700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb2740a3cd0 0x7fb2740a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb284005950 tx=0x7fb2840058e0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.318+0000 7fb28ff24700 1 -- 172.21.15.132:0/2518497206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28806b980 msgr2=0x7fb288199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:15.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.318+0000 7fb28ff24700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28806b980 0x7fb288199610 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb27800e9a0 tx=0x7fb27800ed60 comp rx=0 tx=0).stop 2024-04-23T18:35:15.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.318+0000 7fb28ff24700 1 -- 172.21.15.132:0/2518497206 shutdown_connections 2024-04-23T18:35:15.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.318+0000 7fb28ff24700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb288108a20 0x7fb28819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.318+0000 7fb28ff24700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb28806afe0 0x7fb2881990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.318+0000 7fb28ff24700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fb2740a3cd0 0x7fb2740a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.318+0000 7fb28ff24700 1 --2- 172.21.15.132:0/2518497206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28806b980 0x7fb288199610 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:15.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.318+0000 7fb28ff24700 1 -- 172.21.15.132:0/2518497206 >> 172.21.15.132:0/2518497206 conn(0x7fb2880788b0 msgr2=0x7fb2880ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:15.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.318+0000 7fb28ff24700 1 -- 172.21.15.132:0/2518497206 shutdown_connections 2024-04-23T18:35:15.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:15.318+0000 7fb28ff24700 1 -- 172.21.15.132:0/2518497206 wait complete. 2024-04-23T18:35:15.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:15 smithi148 ceph-mon[32311]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:15.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:15 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1279165620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T18:35:15.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:15 smithi157 ceph-mon[32580]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:15.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:15 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1279165620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T18:35:15.543 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:15 smithi132 ceph-mon[29178]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:15.543 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:15 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1279165620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T18:35:16.288 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:16 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2518497206' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T18:35:16.317 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920827 got 141733920832 for osd.4 2024-04-23T18:35:16.318 DEBUG:teuthology.parallel:result is None 2024-04-23T18:35:16.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:16 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2518497206' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T18:35:16.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:16 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2518497206' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T18:35:16.731 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037798 got 257698037804 for osd.8 2024-04-23T18:35:16.731 DEBUG:teuthology.parallel:result is None 2024-04-23T18:35:16.731 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-23T18:35:16.732 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T18:35:17.087 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:17.464 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:17 smithi132 ceph-mon[29178]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:35:17.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:17 smithi148 ceph-mon[32311]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:35:17.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:17 smithi157 ceph-mon[32580]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:35:18.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.755+0000 7f3feaddc700 1 -- 172.21.15.132:0/1416496923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe4107990 msgr2=0x7f3fe4107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:18.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.755+0000 7f3feaddc700 1 --2- 172.21.15.132:0/1416496923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe4107990 0x7f3fe4107d70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009990 tx=0x7f3fd8009ca0 comp rx=0 tx=0).stop 2024-04-23T18:35:18.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.755+0000 7f3feaddc700 1 -- 172.21.15.132:0/1416496923 shutdown_connections 2024-04-23T18:35:18.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.755+0000 7f3feaddc700 1 --2- 172.21.15.132:0/1416496923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fe4102300 0x7f3fe410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:18.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.755+0000 7f3feaddc700 1 --2- 172.21.15.132:0/1416496923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe4107990 0x7f3fe4107d70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.755+0000 7f3feaddc700 1 --2- 172.21.15.132:0/1416496923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fe4101940 0x7f3fe4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.755+0000 7f3feaddc700 1 -- 172.21.15.132:0/1416496923 >> 172.21.15.132:0/1416496923 conn(0x7f3fe40788f0 msgr2=0x7f3fe4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.755+0000 7f3feaddc700 1 -- 172.21.15.132:0/1416496923 shutdown_connections 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.755+0000 7f3feaddc700 1 -- 172.21.15.132:0/1416496923 wait complete. 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3feaddc700 1 Processor -- start 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3feaddc700 1 -- start start 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3feaddc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe4101940 0x7f3fe419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3feaddc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fe4102300 0x7f3fe419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3feaddc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fe4107990 0x7f3fe41a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3feaddc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe4111bd0 con 0x7f3fe4101940 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3feaddc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe4111a50 con 0x7f3fe4107990 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3feaddc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe4111d50 con 0x7f3fe4102300 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3fe8b78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe4101940 0x7f3fe419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:18.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3fe8b78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe4101940 0x7f3fe419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35246/0 (socket says 172.21.15.132:35246) 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3fe8b78700 1 -- 172.21.15.132:0/23207561 learned_addr learned my addr 172.21.15.132:0/23207561 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3fe3fff700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fe4102300 0x7f3fe419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3fe9379700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fe4107990 0x7f3fe41a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3fe8b78700 1 -- 172.21.15.132:0/23207561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fe4102300 msgr2=0x7f3fe419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3fe8b78700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fe4102300 0x7f3fe419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3fe8b78700 1 -- 172.21.15.132:0/23207561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fe4107990 msgr2=0x7f3fe41a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3fe8b78700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fe4107990 0x7f3fe41a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3fe8b78700 1 -- 172.21.15.132:0/23207561 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe41a5650 con 0x7f3fe4101940 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3fe9379700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fe4107990 0x7f3fe41a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.756+0000 7f3fe8b78700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe4101940 0x7f3fe419a090 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80057f0 tx=0x7f3fd8003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.757+0000 7f3fe1ffb700 1 -- 172.21.15.132:0/23207561 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8027070 con 0x7f3fe4101940 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.757+0000 7f3feaddc700 1 -- 172.21.15.132:0/23207561 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe41a5820 con 0x7f3fe4101940 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.757+0000 7f3fe1ffb700 1 -- 172.21.15.132:0/23207561 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd80041c0 con 0x7f3fe4101940 2024-04-23T18:35:18.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.757+0000 7f3fe1ffb700 1 -- 172.21.15.132:0/23207561 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8022e60 con 0x7f3fe4101940 2024-04-23T18:35:18.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.757+0000 7f3feaddc700 1 -- 172.21.15.132:0/23207561 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe41a5cd0 con 0x7f3fe4101940 2024-04-23T18:35:18.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.758+0000 7f3feaddc700 1 -- 172.21.15.132:0/23207561 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe4109eb0 con 0x7f3fe4101940 2024-04-23T18:35:18.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.759+0000 7f3fe1ffb700 1 -- 172.21.15.132:0/23207561 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3fd802a070 con 0x7f3fe4101940 2024-04-23T18:35:18.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.760+0000 7f3fe1ffb700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3fd40a3cd0 0x7f3fd40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:18.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.760+0000 7f3fe3fff700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3fd40a3cd0 0x7f3fd40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:18.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.760+0000 7f3fe1ffb700 1 -- 172.21.15.132:0/23207561 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f3fd80d1870 con 0x7f3fe4101940 2024-04-23T18:35:18.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.761+0000 7f3fe3fff700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3fd40a3cd0 0x7f3fd40a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0006fd0 tx=0x7f3fd0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:18.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.764+0000 7f3fe1ffb700 1 -- 172.21.15.132:0/23207561 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fd8013070 con 0x7f3fe4101940 2024-04-23T18:35:18.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.983+0000 7f3feaddc700 1 -- 172.21.15.132:0/23207561 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3fe4062e80 con 0x7f3fd40a3cd0 2024-04-23T18:35:18.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.991+0000 7f3fe1ffb700 1 -- 172.21.15.132:0/23207561 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259064 (secure 0 0 0) 0x7f3fe4062e80 con 0x7f3fd40a3cd0 2024-04-23T18:35:18.986 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:18.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.994+0000 7f3feaddc700 1 -- 172.21.15.132:0/23207561 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3fd40a3cd0 msgr2=0x7f3fd40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:18.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.994+0000 7f3feaddc700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3fd40a3cd0 0x7f3fd40a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0006fd0 tx=0x7f3fd0008040 comp rx=0 tx=0).stop 2024-04-23T18:35:18.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.994+0000 7f3feaddc700 1 -- 172.21.15.132:0/23207561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe4101940 msgr2=0x7f3fe419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:18.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.994+0000 7f3feaddc700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe4101940 0x7f3fe419a090 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80057f0 tx=0x7f3fd8003730 comp rx=0 tx=0).stop 2024-04-23T18:35:18.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.994+0000 7f3feaddc700 1 -- 172.21.15.132:0/23207561 shutdown_connections 2024-04-23T18:35:18.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.994+0000 7f3feaddc700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fe4107990 0x7f3fe41a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:18.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.994+0000 7f3feaddc700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3fd40a3cd0 0x7f3fd40a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:18.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.994+0000 7f3feaddc700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe4101940 0x7f3fe419a090 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:18.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.994+0000 7f3feaddc700 1 --2- 172.21.15.132:0/23207561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fe4102300 0x7f3fe419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:18.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.994+0000 7f3feaddc700 1 -- 172.21.15.132:0/23207561 >> 172.21.15.132:0/23207561 conn(0x7f3fe40788f0 msgr2=0x7f3fe4100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:18.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.994+0000 7f3feaddc700 1 -- 172.21.15.132:0/23207561 shutdown_connections 2024-04-23T18:35:18.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:18.994+0000 7f3feaddc700 1 -- 172.21.15.132:0/23207561 wait complete. 2024-04-23T18:35:18.989 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-04-23T18:35:19.359 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:19 smithi132 ceph-mon[29178]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:19.359 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:19 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:19.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:19 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:19.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:19 smithi148 ceph-mon[32311]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:19.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:19 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:19.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:19 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:19.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:19 smithi157 ceph-mon[32580]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:19.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:19 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:19.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:19 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:19.763 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-04-23T18:35:18.145553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657271,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":866,"num_read_kb":556,"num_write":419,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352752,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589072,"statfs":{"total":1151940427776,"available":1151579209728,"internally_reserved":0,"allocated":12128256,"data_stored":7504917,"data_compressed":182772,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003295"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253272+0000","last_change":"2024-04-23T18:34:01.231204+0000","last_active":"2024-04-23T18:34:55.253272+0000","last_peered":"2024-04-23T18:34:55.253272+0000","last_clean":"2024-04-23T18:34:55.253272+0000","last_became_active":"2024-04-23T18:34:01.230836+0000","last_became_peered":"2024-04-23T18:34:01.230836+0000","last_unstale":"2024-04-23T18:34:55.253272+0000","last_undegraded":"2024-04-23T18:34:55.253272+0000","last_fullsized":"2024-04-23T18:34:55.253272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:57.744807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256840+0000","last_change":"2024-04-23T18:34:01.225833+0000","last_active":"2024-04-23T18:34:55.256840+0000","last_peered":"2024-04-23T18:34:55.256840+0000","last_clean":"2024-04-23T18:34:55.256840+0000","last_became_active":"2024-04-23T18:34:01.225098+0000","last_became_peered":"2024-04-23T18:34:01.225098+0000","last_unstale":"2024-04-23T18:34:55.256840+0000","last_undegraded":"2024-04-23T18:34:55.256840+0000","last_fullsized":"2024-04-23T18:34:55.256840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:20.720137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256596+0000","last_change":"2024-04-23T18:34:01.235269+0000","last_active":"2024-04-23T18:34:55.256596+0000","last_peered":"2024-04-23T18:34:55.256596+0000","last_clean":"2024-04-23T18:34:55.256596+0000","last_became_active":"2024-04-23T18:34:01.234921+0000","last_became_peered":"2024-04-23T18:34:01.234921+0000","last_unstale":"2024-04-23T18:34:55.256596+0000","last_undegraded":"2024-04-23T18:34:55.256596+0000","last_fullsized":"2024-04-23T18:34:55.256596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:06:20.645693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.287914+0000","last_change":"2024-04-23T18:34:01.233316+0000","last_active":"2024-04-23T18:34:55.287914+0000","last_peered":"2024-04-23T18:34:55.287914+0000","last_clean":"2024-04-23T18:34:55.287914+0000","last_became_active":"2024-04-23T18:34:01.231636+0000","last_became_peered":"2024-04-23T18:34:01.231636+0000","last_unstale":"2024-04-23T18:34:55.287914+0000","last_undegraded":"2024-04-23T18:34:55.287914+0000","last_fullsized":"2024-04-23T18:34:55.287914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:15.837129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255885+0000","last_change":"2024-04-23T18:34:01.228073+0000","last_active":"2024-04-23T18:34:55.255885+0000","last_peered":"2024-04-23T18:34:55.255885+0000","last_clean":"2024-04-23T18:34:55.255885+0000","last_became_active":"2024-04-23T18:34:01.227701+0000","last_became_peered":"2024-04-23T18:34:01.227701+0000","last_unstale":"2024-04-23T18:34:55.255885+0000","last_undegraded":"2024-04-23T18:34:55.255885+0000","last_fullsized":"2024-04-23T18:34:55.255885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:17:26.556498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258233+0000","last_change":"2024-04-23T18:34:55.258233+0000","last_active":"2024-04-23T18:34:55.258233+0000","last_peered":"2024-04-23T18:34:55.258233+0000","last_clean":"2024-04-23T18:34:55.258233+0000","last_became_active":"2024-04-23T18:34:55.257836+0000","last_became_peered":"2024-04-23T18:34:55.257836+0000","last_unstale":"2024-04-23T18:34:55.258233+0000","last_undegraded":"2024-04-23T18:34:55.258233+0000","last_fullsized":"2024-04-23T18:34:55.258233+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:20.279369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260737+0000","last_change":"2024-04-23T18:34:55.260737+0000","last_active":"2024-04-23T18:34:55.260737+0000","last_peered":"2024-04-23T18:34:55.260737+0000","last_clean":"2024-04-23T18:34:55.260737+0000","last_became_active":"2024-04-23T18:34:55.260231+0000","last_became_peered":"2024-04-23T18:34:55.260231+0000","last_unstale":"2024-04-23T18:34:55.260737+0000","last_undegraded":"2024-04-23T18:34:55.260737+0000","last_fullsized":"2024-04-23T18:34:55.260737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:45.292690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.259963+0000","last_change":"2024-04-23T18:34:55.259963+0000","last_active":"2024-04-23T18:34:55.259963+0000","last_peered":"2024-04-23T18:34:55.259963+0000","last_clean":"2024-04-23T18:34:55.259963+0000","last_became_active":"2024-04-23T18:34:55.259342+0000","last_became_peered":"2024-04-23T18:34:55.259342+0000","last_unstale":"2024-04-23T18:34:55.259963+0000","last_undegraded":"2024-04-23T18:34:55.259963+0000","last_fullsized":"2024-04-23T18:34:55.259963+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:42:36.612809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286947+0000","last_change":"2024-04-23T18:34:01.234094+0000","last_active":"2024-04-23T18:34:55.286947+0000","last_peered":"2024-04-23T18:34:55.286947+0000","last_clean":"2024-04-23T18:34:55.286947+0000","last_became_active":"2024-04-23T18:34:01.232499+0000","last_became_peered":"2024-04-23T18:34:01.232499+0000","last_unstale":"2024-04-23T18:34:55.286947+0000","last_undegraded":"2024-04-23T18:34:55.286947+0000","last_fullsized":"2024-04-23T18:34:55.286947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:33.004245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253537+0000","last_change":"2024-04-23T18:34:01.229175+0000","last_active":"2024-04-23T18:34:55.253537+0000","last_peered":"2024-04-23T18:34:55.253537+0000","last_clean":"2024-04-23T18:34:55.253537+0000","last_became_active":"2024-04-23T18:34:01.228837+0000","last_became_peered":"2024-04-23T18:34:01.228837+0000","last_unstale":"2024-04-23T18:34:55.253537+0000","last_undegraded":"2024-04-23T18:34:55.253537+0000","last_fullsized":"2024-04-23T18:34:55.253537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:57.625110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258853+0000","last_change":"2024-04-23T18:34:01.235557+0000","last_active":"2024-04-23T18:34:55.258853+0000","last_peered":"2024-04-23T18:34:55.258853+0000","last_clean":"2024-04-23T18:34:55.258853+0000","last_became_active":"2024-04-23T18:34:01.235184+0000","last_became_peered":"2024-04-23T18:34:01.235184+0000","last_unstale":"2024-04-23T18:34:55.258853+0000","last_undegraded":"2024-04-23T18:34:55.258853+0000","last_fullsized":"2024-04-23T18:34:55.258853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:37:27.138041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.287745+0000","last_change":"2024-04-23T18:34:01.234070+0000","last_active":"2024-04-23T18:34:55.287745+0000","last_peered":"2024-04-23T18:34:55.287745+0000","last_clean":"2024-04-23T18:34:55.287745+0000","last_became_active":"2024-04-23T18:34:01.233727+0000","last_became_peered":"2024-04-23T18:34:01.233727+0000","last_unstale":"2024-04-23T18:34:55.287745+0000","last_undegraded":"2024-04-23T18:34:55.287745+0000","last_fullsized":"2024-04-23T18:34:55.287745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:59:16.489819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256003+0000","last_change":"2024-04-23T18:34:01.228256+0000","last_active":"2024-04-23T18:34:55.256003+0000","last_peered":"2024-04-23T18:34:55.256003+0000","last_clean":"2024-04-23T18:34:55.256003+0000","last_became_active":"2024-04-23T18:34:01.224400+0000","last_became_peered":"2024-04-23T18:34:01.224400+0000","last_unstale":"2024-04-23T18:34:55.256003+0000","last_undegraded":"2024-04-23T18:34:55.256003+0000","last_fullsized":"2024-04-23T18:34:55.256003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:37:28.401695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256371+0000","last_change":"2024-04-23T18:34:01.227250+0000","last_active":"2024-04-23T18:34:55.256371+0000","last_peered":"2024-04-23T18:34:55.256371+0000","last_clean":"2024-04-23T18:34:55.256371+0000","last_became_active":"2024-04-23T18:34:01.226937+0000","last_became_peered":"2024-04-23T18:34:01.226937+0000","last_unstale":"2024-04-23T18:34:55.256371+0000","last_undegraded":"2024-04-23T18:34:55.256371+0000","last_fullsized":"2024-04-23T18:34:55.256371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:26:20.801691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256119+0000","last_change":"2024-04-23T18:34:01.232351+0000","last_active":"2024-04-23T18:34:55.256119+0000","last_peered":"2024-04-23T18:34:55.256119+0000","last_clean":"2024-04-23T18:34:55.256119+0000","last_became_active":"2024-04-23T18:34:01.231987+0000","last_became_peered":"2024-04-23T18:34:01.231987+0000","last_unstale":"2024-04-23T18:34:55.256119+0000","last_undegraded":"2024-04-23T18:34:55.256119+0000","last_fullsized":"2024-04-23T18:34:55.256119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:50.886496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286235+0000","last_change":"2024-04-23T18:34:01.232196+0000","last_active":"2024-04-23T18:34:55.286235+0000","last_peered":"2024-04-23T18:34:55.286235+0000","last_clean":"2024-04-23T18:34:55.286235+0000","last_became_active":"2024-04-23T18:34:01.231746+0000","last_became_peered":"2024-04-23T18:34:01.231746+0000","last_unstale":"2024-04-23T18:34:55.286235+0000","last_undegraded":"2024-04-23T18:34:55.286235+0000","last_fullsized":"2024-04-23T18:34:55.286235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:58.888549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253346+0000","last_change":"2024-04-23T18:34:01.231509+0000","last_active":"2024-04-23T18:34:55.253346+0000","last_peered":"2024-04-23T18:34:55.253346+0000","last_clean":"2024-04-23T18:34:55.253346+0000","last_became_active":"2024-04-23T18:34:01.231017+0000","last_became_peered":"2024-04-23T18:34:01.231017+0000","last_unstale":"2024-04-23T18:34:55.253346+0000","last_undegraded":"2024-04-23T18:34:55.253346+0000","last_fullsized":"2024-04-23T18:34:55.253346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:04:51.929462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255975+0000","last_change":"2024-04-23T18:34:01.223285+0000","last_active":"2024-04-23T18:34:55.255975+0000","last_peered":"2024-04-23T18:34:55.255975+0000","last_clean":"2024-04-23T18:34:55.255975+0000","last_became_active":"2024-04-23T18:34:01.222962+0000","last_became_peered":"2024-04-23T18:34:01.222962+0000","last_unstale":"2024-04-23T18:34:55.255975+0000","last_undegraded":"2024-04-23T18:34:55.255975+0000","last_fullsized":"2024-04-23T18:34:55.255975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:18.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255490+0000","last_change":"2024-04-23T18:34:01.225922+0000","last_active":"2024-04-23T18:34:55.255490+0000","last_peered":"2024-04-23T18:34:55.255490+0000","last_clean":"2024-04-23T18:34:55.255490+0000","last_became_active":"2024-04-23T18:34:01.225235+0000","last_became_peered":"2024-04-23T18:34:01.225235+0000","last_unstale":"2024-04-23T18:34:55.255490+0000","last_undegraded":"2024-04-23T18:34:55.255490+0000","last_fullsized":"2024-04-23T18:34:55.255490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:33.480782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:12.629484+0000","last_change":"2024-04-23T18:34:00.224606+0000","last_active":"2024-04-23T18:35:12.629484+0000","last_peered":"2024-04-23T18:35:12.629484+0000","last_clean":"2024-04-23T18:35:12.629484+0000","last_became_active":"2024-04-23T18:34:00.224291+0000","last_became_peered":"2024-04-23T18:34:00.224291+0000","last_unstale":"2024-04-23T18:35:12.629484+0000","last_undegraded":"2024-04-23T18:35:12.629484+0000","last_fullsized":"2024-04-23T18:35:12.629484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:40.734704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286232+0000","last_change":"2024-04-23T18:34:01.235023+0000","last_active":"2024-04-23T18:34:55.286232+0000","last_peered":"2024-04-23T18:34:55.286232+0000","last_clean":"2024-04-23T18:34:55.286232+0000","last_became_active":"2024-04-23T18:34:01.234672+0000","last_became_peered":"2024-04-23T18:34:01.234672+0000","last_unstale":"2024-04-23T18:34:55.286232+0000","last_undegraded":"2024-04-23T18:34:55.286232+0000","last_fullsized":"2024-04-23T18:34:55.286232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:33.734343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286517+0000","last_change":"2024-04-23T18:34:00.232552+0000","last_active":"2024-04-23T18:34:55.286517+0000","last_peered":"2024-04-23T18:34:55.286517+0000","last_clean":"2024-04-23T18:34:55.286517+0000","last_became_active":"2024-04-23T18:34:00.232226+0000","last_became_peered":"2024-04-23T18:34:00.232226+0000","last_unstale":"2024-04-23T18:34:55.286517+0000","last_undegraded":"2024-04-23T18:34:55.286517+0000","last_fullsized":"2024-04-23T18:34:55.286517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.563757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.257141+0000","last_change":"2024-04-23T18:34:01.236013+0000","last_active":"2024-04-23T18:34:55.257141+0000","last_peered":"2024-04-23T18:34:55.257141+0000","last_clean":"2024-04-23T18:34:55.257141+0000","last_became_active":"2024-04-23T18:34:01.235590+0000","last_became_peered":"2024-04-23T18:34:01.235590+0000","last_unstale":"2024-04-23T18:34:55.257141+0000","last_undegraded":"2024-04-23T18:34:55.257141+0000","last_fullsized":"2024-04-23T18:34:55.257141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:45.741648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.298010+0000","last_change":"2024-04-23T18:34:55.298010+0000","last_active":"2024-04-23T18:34:55.298010+0000","last_peered":"2024-04-23T18:34:55.298010+0000","last_clean":"2024-04-23T18:34:55.298010+0000","last_became_active":"2024-04-23T18:34:55.288221+0000","last_became_peered":"2024-04-23T18:34:55.288221+0000","last_unstale":"2024-04-23T18:34:55.298010+0000","last_undegraded":"2024-04-23T18:34:55.298010+0000","last_fullsized":"2024-04-23T18:34:55.298010+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:21:35.469676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286457+0000","last_change":"2024-04-23T18:34:01.230435+0000","last_active":"2024-04-23T18:34:55.286457+0000","last_peered":"2024-04-23T18:34:55.286457+0000","last_clean":"2024-04-23T18:34:55.286457+0000","last_became_active":"2024-04-23T18:34:01.230095+0000","last_became_peered":"2024-04-23T18:34:01.230095+0000","last_unstale":"2024-04-23T18:34:55.286457+0000","last_undegraded":"2024-04-23T18:34:55.286457+0000","last_fullsized":"2024-04-23T18:34:55.286457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:31.653254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256171+0000","last_change":"2024-04-23T18:34:00.227853+0000","last_active":"2024-04-23T18:34:55.256171+0000","last_peered":"2024-04-23T18:34:55.256171+0000","last_clean":"2024-04-23T18:34:55.256171+0000","last_became_active":"2024-04-23T18:34:00.226596+0000","last_became_peered":"2024-04-23T18:34:00.226596+0000","last_unstale":"2024-04-23T18:34:55.256171+0000","last_undegraded":"2024-04-23T18:34:55.256171+0000","last_fullsized":"2024-04-23T18:34:55.256171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:16:10.077969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260701+0000","last_change":"2024-04-23T18:34:01.235955+0000","last_active":"2024-04-23T18:34:55.260701+0000","last_peered":"2024-04-23T18:34:55.260701+0000","last_clean":"2024-04-23T18:34:55.260701+0000","last_became_active":"2024-04-23T18:34:01.235609+0000","last_became_peered":"2024-04-23T18:34:01.235609+0000","last_unstale":"2024-04-23T18:34:55.260701+0000","last_undegraded":"2024-04-23T18:34:55.260701+0000","last_fullsized":"2024-04-23T18:34:55.260701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:07:39.480213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:12.615841+0000","last_change":"2024-04-23T18:34:00.226319+0000","last_active":"2024-04-23T18:35:12.615841+0000","last_peered":"2024-04-23T18:35:12.615841+0000","last_clean":"2024-04-23T18:35:12.615841+0000","last_became_active":"2024-04-23T18:34:00.225796+0000","last_became_peered":"2024-04-23T18:34:00.225796+0000","last_unstale":"2024-04-23T18:35:12.615841+0000","last_undegraded":"2024-04-23T18:35:12.615841+0000","last_fullsized":"2024-04-23T18:35:12.615841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:11:18.309181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255891+0000","last_change":"2024-04-23T18:34:01.227336+0000","last_active":"2024-04-23T18:34:55.255891+0000","last_peered":"2024-04-23T18:34:55.255891+0000","last_clean":"2024-04-23T18:34:55.255891+0000","last_became_active":"2024-04-23T18:34:01.226987+0000","last_became_peered":"2024-04-23T18:34:01.226987+0000","last_unstale":"2024-04-23T18:34:55.255891+0000","last_undegraded":"2024-04-23T18:34:55.255891+0000","last_fullsized":"2024-04-23T18:34:55.255891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:22:30.058673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.294322+0000","last_change":"2024-04-23T18:34:55.294322+0000","last_active":"2024-04-23T18:34:55.294322+0000","last_peered":"2024-04-23T18:34:55.294322+0000","last_clean":"2024-04-23T18:34:55.294322+0000","last_became_active":"2024-04-23T18:34:55.288855+0000","last_became_peered":"2024-04-23T18:34:55.288855+0000","last_unstale":"2024-04-23T18:34:55.294322+0000","last_undegraded":"2024-04-23T18:34:55.294322+0000","last_fullsized":"2024-04-23T18:34:55.294322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:07:50.097696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.287339+0000","last_change":"2024-04-23T18:34:55.287339+0000","last_active":"2024-04-23T18:34:55.287339+0000","last_peered":"2024-04-23T18:34:55.287339+0000","last_clean":"2024-04-23T18:34:55.287339+0000","last_became_active":"2024-04-23T18:34:55.287062+0000","last_became_peered":"2024-04-23T18:34:55.287062+0000","last_unstale":"2024-04-23T18:34:55.287339+0000","last_undegraded":"2024-04-23T18:34:55.287339+0000","last_fullsized":"2024-04-23T18:34:55.287339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:44.864600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255616+0000","last_change":"2024-04-23T18:34:00.227595+0000","last_active":"2024-04-23T18:34:55.255616+0000","last_peered":"2024-04-23T18:34:55.255616+0000","last_clean":"2024-04-23T18:34:55.255616+0000","last_became_active":"2024-04-23T18:34:00.226830+0000","last_became_peered":"2024-04-23T18:34:00.226830+0000","last_unstale":"2024-04-23T18:34:55.255616+0000","last_undegraded":"2024-04-23T18:34:55.255616+0000","last_fullsized":"2024-04-23T18:34:55.255616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:20.661984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286109+0000","last_change":"2024-04-23T18:34:01.225053+0000","last_active":"2024-04-23T18:34:55.286109+0000","last_peered":"2024-04-23T18:34:55.286109+0000","last_clean":"2024-04-23T18:34:55.286109+0000","last_became_active":"2024-04-23T18:34:01.224723+0000","last_became_peered":"2024-04-23T18:34:01.224723+0000","last_unstale":"2024-04-23T18:34:55.286109+0000","last_undegraded":"2024-04-23T18:34:55.286109+0000","last_fullsized":"2024-04-23T18:34:55.286109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:01:51.486691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260492+0000","last_change":"2024-04-23T18:34:00.224783+0000","last_active":"2024-04-23T18:34:55.260492+0000","last_peered":"2024-04-23T18:34:55.260492+0000","last_clean":"2024-04-23T18:34:55.260492+0000","last_became_active":"2024-04-23T18:34:00.224436+0000","last_became_peered":"2024-04-23T18:34:00.224436+0000","last_unstale":"2024-04-23T18:34:55.260492+0000","last_undegraded":"2024-04-23T18:34:55.260492+0000","last_fullsized":"2024-04-23T18:34:55.260492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:33.829188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.254798+0000","last_change":"2024-04-23T18:34:01.224283+0000","last_active":"2024-04-23T18:34:55.254798+0000","last_peered":"2024-04-23T18:34:55.254798+0000","last_clean":"2024-04-23T18:34:55.254798+0000","last_became_active":"2024-04-23T18:34:01.223949+0000","last_became_peered":"2024-04-23T18:34:01.223949+0000","last_unstale":"2024-04-23T18:34:55.254798+0000","last_undegraded":"2024-04-23T18:34:55.254798+0000","last_fullsized":"2024-04-23T18:34:55.254798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:50.560687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":161,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:14.617770+0000","last_change":"2024-04-23T18:34:00.218453+0000","last_active":"2024-04-23T18:35:14.617770+0000","last_peered":"2024-04-23T18:35:14.617770+0000","last_clean":"2024-04-23T18:35:14.617770+0000","last_became_active":"2024-04-23T18:34:00.218036+0000","last_became_peered":"2024-04-23T18:34:00.218036+0000","last_unstale":"2024-04-23T18:35:14.617770+0000","last_undegraded":"2024-04-23T18:35:14.617770+0000","last_fullsized":"2024-04-23T18:35:14.617770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:49.485185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286939+0000","last_change":"2024-04-23T18:34:01.231143+0000","last_active":"2024-04-23T18:34:55.286939+0000","last_peered":"2024-04-23T18:34:55.286939+0000","last_clean":"2024-04-23T18:34:55.286939+0000","last_became_active":"2024-04-23T18:34:01.230812+0000","last_became_peered":"2024-04-23T18:34:01.230812+0000","last_unstale":"2024-04-23T18:34:55.286939+0000","last_undegraded":"2024-04-23T18:34:55.286939+0000","last_fullsized":"2024-04-23T18:34:55.286939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:28.704807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258862+0000","last_change":"2024-04-23T18:34:00.235711+0000","last_active":"2024-04-23T18:34:55.258862+0000","last_peered":"2024-04-23T18:34:55.258862+0000","last_clean":"2024-04-23T18:34:55.258862+0000","last_became_active":"2024-04-23T18:34:00.235333+0000","last_became_peered":"2024-04-23T18:34:00.235333+0000","last_unstale":"2024-04-23T18:34:55.258862+0000","last_undegraded":"2024-04-23T18:34:55.258862+0000","last_fullsized":"2024-04-23T18:34:55.258862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:51.320744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.257055+0000","last_change":"2024-04-23T18:34:01.238291+0000","last_active":"2024-04-23T18:34:55.257055+0000","last_peered":"2024-04-23T18:34:55.257055+0000","last_clean":"2024-04-23T18:34:55.257055+0000","last_became_active":"2024-04-23T18:34:01.237972+0000","last_became_peered":"2024-04-23T18:34:01.237972+0000","last_unstale":"2024-04-23T18:34:55.257055+0000","last_undegraded":"2024-04-23T18:34:55.257055+0000","last_fullsized":"2024-04-23T18:34:55.257055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:36:17.108170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260912+0000","last_change":"2024-04-23T18:34:00.227249+0000","last_active":"2024-04-23T18:34:55.260912+0000","last_peered":"2024-04-23T18:34:55.260912+0000","last_clean":"2024-04-23T18:34:55.260912+0000","last_became_active":"2024-04-23T18:34:00.226854+0000","last_became_peered":"2024-04-23T18:34:00.226854+0000","last_unstale":"2024-04-23T18:34:55.260912+0000","last_undegraded":"2024-04-23T18:34:55.260912+0000","last_fullsized":"2024-04-23T18:34:55.260912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:49.619906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286572+0000","last_change":"2024-04-23T18:34:01.225518+0000","last_active":"2024-04-23T18:34:55.286572+0000","last_peered":"2024-04-23T18:34:55.286572+0000","last_clean":"2024-04-23T18:34:55.286572+0000","last_became_active":"2024-04-23T18:34:01.224907+0000","last_became_peered":"2024-04-23T18:34:01.224907+0000","last_unstale":"2024-04-23T18:34:55.286572+0000","last_undegraded":"2024-04-23T18:34:55.286572+0000","last_fullsized":"2024-04-23T18:34:55.286572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:19.876629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":196,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:14.617719+0000","last_change":"2024-04-23T18:34:00.224770+0000","last_active":"2024-04-23T18:35:14.617719+0000","last_peered":"2024-04-23T18:35:14.617719+0000","last_clean":"2024-04-23T18:35:14.617719+0000","last_became_active":"2024-04-23T18:34:00.224395+0000","last_became_peered":"2024-04-23T18:34:00.224395+0000","last_unstale":"2024-04-23T18:35:14.617719+0000","last_undegraded":"2024-04-23T18:35:14.617719+0000","last_fullsized":"2024-04-23T18:35:14.617719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:14:09.172479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":148,"num_read_kb":132,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256208+0000","last_change":"2024-04-23T18:34:01.227771+0000","last_active":"2024-04-23T18:34:55.256208+0000","last_peered":"2024-04-23T18:34:55.256208+0000","last_clean":"2024-04-23T18:34:55.256208+0000","last_became_active":"2024-04-23T18:34:01.225063+0000","last_became_peered":"2024-04-23T18:34:01.225063+0000","last_unstale":"2024-04-23T18:34:55.256208+0000","last_undegraded":"2024-04-23T18:34:55.256208+0000","last_fullsized":"2024-04-23T18:34:55.256208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:53.921587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255405+0000","last_change":"2024-04-23T18:34:00.230455+0000","last_active":"2024-04-23T18:34:55.255405+0000","last_peered":"2024-04-23T18:34:55.255405+0000","last_clean":"2024-04-23T18:34:55.255405+0000","last_became_active":"2024-04-23T18:34:00.229934+0000","last_became_peered":"2024-04-23T18:34:00.229934+0000","last_unstale":"2024-04-23T18:34:55.255405+0000","last_undegraded":"2024-04-23T18:34:55.255405+0000","last_fullsized":"2024-04-23T18:34:55.255405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:26:34.787606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256945+0000","last_change":"2024-04-23T18:34:01.225757+0000","last_active":"2024-04-23T18:34:55.256945+0000","last_peered":"2024-04-23T18:34:55.256945+0000","last_clean":"2024-04-23T18:34:55.256945+0000","last_became_active":"2024-04-23T18:34:01.225447+0000","last_became_peered":"2024-04-23T18:34:01.225447+0000","last_unstale":"2024-04-23T18:34:55.256945+0000","last_undegraded":"2024-04-23T18:34:55.256945+0000","last_fullsized":"2024-04-23T18:34:55.256945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:17.212135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260447+0000","last_change":"2024-04-23T18:34:00.231749+0000","last_active":"2024-04-23T18:34:55.260447+0000","last_peered":"2024-04-23T18:34:55.260447+0000","last_clean":"2024-04-23T18:34:55.260447+0000","last_became_active":"2024-04-23T18:34:00.231207+0000","last_became_peered":"2024-04-23T18:34:00.231207+0000","last_unstale":"2024-04-23T18:34:55.260447+0000","last_undegraded":"2024-04-23T18:34:55.260447+0000","last_fullsized":"2024-04-23T18:34:55.260447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:23:05.723265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258587+0000","last_change":"2024-04-23T18:34:01.241116+0000","last_active":"2024-04-23T18:34:55.258587+0000","last_peered":"2024-04-23T18:34:55.258587+0000","last_clean":"2024-04-23T18:34:55.258587+0000","last_became_active":"2024-04-23T18:34:01.240785+0000","last_became_peered":"2024-04-23T18:34:01.240785+0000","last_unstale":"2024-04-23T18:34:55.258587+0000","last_undegraded":"2024-04-23T18:34:55.258587+0000","last_fullsized":"2024-04-23T18:34:55.258587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:48.323241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:13.638111+0000","last_change":"2024-04-23T18:34:00.228578+0000","last_active":"2024-04-23T18:35:13.638111+0000","last_peered":"2024-04-23T18:35:13.638111+0000","last_clean":"2024-04-23T18:35:13.638111+0000","last_became_active":"2024-04-23T18:34:00.228238+0000","last_became_peered":"2024-04-23T18:34:00.228238+0000","last_unstale":"2024-04-23T18:35:13.638111+0000","last_undegraded":"2024-04-23T18:35:13.638111+0000","last_fullsized":"2024-04-23T18:35:13.638111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:58:21.283593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.257726+0000","last_change":"2024-04-23T18:34:01.239481+0000","last_active":"2024-04-23T18:34:55.257726+0000","last_peered":"2024-04-23T18:34:55.257726+0000","last_clean":"2024-04-23T18:34:55.257726+0000","last_became_active":"2024-04-23T18:34:01.239042+0000","last_became_peered":"2024-04-23T18:34:01.239042+0000","last_unstale":"2024-04-23T18:34:55.257726+0000","last_undegraded":"2024-04-23T18:34:55.257726+0000","last_fullsized":"2024-04-23T18:34:55.257726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:45.755462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.273948+0000","last_change":"2024-04-23T18:34:55.273529+0000","last_active":"2024-04-23T18:34:55.273948+0000","last_peered":"2024-04-23T18:34:55.273948+0000","last_clean":"2024-04-23T18:34:55.273948+0000","last_became_active":"2024-04-23T18:34:55.259780+0000","last_became_peered":"2024-04-23T18:34:55.259780+0000","last_unstale":"2024-04-23T18:34:55.273948+0000","last_undegraded":"2024-04-23T18:34:55.273948+0000","last_fullsized":"2024-04-23T18:34:55.273948+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:48:56.272606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260773+0000","last_change":"2024-04-23T18:34:01.239943+0000","last_active":"2024-04-23T18:34:55.260773+0000","last_peered":"2024-04-23T18:34:55.260773+0000","last_clean":"2024-04-23T18:34:55.260773+0000","last_became_active":"2024-04-23T18:34:01.239588+0000","last_became_peered":"2024-04-23T18:34:01.239588+0000","last_unstale":"2024-04-23T18:34:55.260773+0000","last_undegraded":"2024-04-23T18:34:55.260773+0000","last_fullsized":"2024-04-23T18:34:55.260773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:51:39.344341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256091+0000","last_change":"2024-04-23T18:34:00.227372+0000","last_active":"2024-04-23T18:34:55.256091+0000","last_peered":"2024-04-23T18:34:55.256091+0000","last_clean":"2024-04-23T18:34:55.256091+0000","last_became_active":"2024-04-23T18:34:00.226709+0000","last_became_peered":"2024-04-23T18:34:00.226709+0000","last_unstale":"2024-04-23T18:34:55.256091+0000","last_undegraded":"2024-04-23T18:34:55.256091+0000","last_fullsized":"2024-04-23T18:34:55.256091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:20.047108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253087+0000","last_change":"2024-04-23T18:34:01.235389+0000","last_active":"2024-04-23T18:34:55.253087+0000","last_peered":"2024-04-23T18:34:55.253087+0000","last_clean":"2024-04-23T18:34:55.253087+0000","last_became_active":"2024-04-23T18:34:01.235026+0000","last_became_peered":"2024-04-23T18:34:01.235026+0000","last_unstale":"2024-04-23T18:34:55.253087+0000","last_undegraded":"2024-04-23T18:34:55.253087+0000","last_fullsized":"2024-04-23T18:34:55.253087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:48.849525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260852+0000","last_change":"2024-04-23T18:34:55.260852+0000","last_active":"2024-04-23T18:34:55.260852+0000","last_peered":"2024-04-23T18:34:55.260852+0000","last_clean":"2024-04-23T18:34:55.260852+0000","last_became_active":"2024-04-23T18:34:55.260452+0000","last_became_peered":"2024-04-23T18:34:55.260452+0000","last_unstale":"2024-04-23T18:34:55.260852+0000","last_undegraded":"2024-04-23T18:34:55.260852+0000","last_fullsized":"2024-04-23T18:34:55.260852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:28:13.496447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253435+0000","last_change":"2024-04-23T18:34:01.224992+0000","last_active":"2024-04-23T18:34:55.253435+0000","last_peered":"2024-04-23T18:34:55.253435+0000","last_clean":"2024-04-23T18:34:55.253435+0000","last_became_active":"2024-04-23T18:34:01.224508+0000","last_became_peered":"2024-04-23T18:34:01.224508+0000","last_unstale":"2024-04-23T18:34:55.253435+0000","last_undegraded":"2024-04-23T18:34:55.253435+0000","last_fullsized":"2024-04-23T18:34:55.253435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:24.183451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:15.634122+0000","last_change":"2024-04-23T18:34:00.234818+0000","last_active":"2024-04-23T18:35:15.634122+0000","last_peered":"2024-04-23T18:35:15.634122+0000","last_clean":"2024-04-23T18:35:15.634122+0000","last_became_active":"2024-04-23T18:34:00.234329+0000","last_became_peered":"2024-04-23T18:34:00.234329+0000","last_unstale":"2024-04-23T18:35:15.634122+0000","last_undegraded":"2024-04-23T18:35:15.634122+0000","last_fullsized":"2024-04-23T18:35:15.634122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:43:02.405694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.261073+0000","last_change":"2024-04-23T18:34:55.261073+0000","last_active":"2024-04-23T18:34:55.261073+0000","last_peered":"2024-04-23T18:34:55.261073+0000","last_clean":"2024-04-23T18:34:55.261073+0000","last_became_active":"2024-04-23T18:34:55.260655+0000","last_became_peered":"2024-04-23T18:34:55.260655+0000","last_unstale":"2024-04-23T18:34:55.261073+0000","last_undegraded":"2024-04-23T18:34:55.261073+0000","last_fullsized":"2024-04-23T18:34:55.261073+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:37:36.797785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260771+0000","last_change":"2024-04-23T18:34:01.234184+0000","last_active":"2024-04-23T18:34:55.260771+0000","last_peered":"2024-04-23T18:34:55.260771+0000","last_clean":"2024-04-23T18:34:55.260771+0000","last_became_active":"2024-04-23T18:34:01.233846+0000","last_became_peered":"2024-04-23T18:34:01.233846+0000","last_unstale":"2024-04-23T18:34:55.260771+0000","last_undegraded":"2024-04-23T18:34:55.260771+0000","last_fullsized":"2024-04-23T18:34:55.260771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:41.771495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:13.637122+0000","last_change":"2024-04-23T18:34:00.227862+0000","last_active":"2024-04-23T18:35:13.637122+0000","last_peered":"2024-04-23T18:35:13.637122+0000","last_clean":"2024-04-23T18:35:13.637122+0000","last_became_active":"2024-04-23T18:34:00.227515+0000","last_became_peered":"2024-04-23T18:34:00.227515+0000","last_unstale":"2024-04-23T18:35:13.637122+0000","last_undegraded":"2024-04-23T18:35:13.637122+0000","last_fullsized":"2024-04-23T18:35:13.637122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:18.595690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286382+0000","last_change":"2024-04-23T18:34:01.236338+0000","last_active":"2024-04-23T18:34:55.286382+0000","last_peered":"2024-04-23T18:34:55.286382+0000","last_clean":"2024-04-23T18:34:55.286382+0000","last_became_active":"2024-04-23T18:34:01.235795+0000","last_became_peered":"2024-04-23T18:34:01.235795+0000","last_unstale":"2024-04-23T18:34:55.286382+0000","last_undegraded":"2024-04-23T18:34:55.286382+0000","last_fullsized":"2024-04-23T18:34:55.286382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:45:01.928793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.287511+0000","last_change":"2024-04-23T18:34:00.221851+0000","last_active":"2024-04-23T18:34:55.287511+0000","last_peered":"2024-04-23T18:34:55.287511+0000","last_clean":"2024-04-23T18:34:55.287511+0000","last_became_active":"2024-04-23T18:34:00.221513+0000","last_became_peered":"2024-04-23T18:34:00.221513+0000","last_unstale":"2024-04-23T18:34:55.287511+0000","last_undegraded":"2024-04-23T18:34:55.287511+0000","last_fullsized":"2024-04-23T18:34:55.287511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:15:04.608759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.257868+0000","last_change":"2024-04-23T18:32:48.218931+0000","last_active":"2024-04-23T18:34:55.257868+0000","last_peered":"2024-04-23T18:34:55.257868+0000","last_clean":"2024-04-23T18:34:55.257868+0000","last_became_active":"2024-04-23T18:32:48.218484+0000","last_became_peered":"2024-04-23T18:32:48.218484+0000","last_unstale":"2024-04-23T18:34:55.257868+0000","last_undegraded":"2024-04-23T18:34:55.257868+0000","last_fullsized":"2024-04-23T18:34:55.257868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_clean_scrub_stamp":"2024-04-23T18:29:21.029319+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:03:42.234165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255404+0000","last_change":"2024-04-23T18:34:01.220346+0000","last_active":"2024-04-23T18:34:55.255404+0000","last_peered":"2024-04-23T18:34:55.255404+0000","last_clean":"2024-04-23T18:34:55.255404+0000","last_became_active":"2024-04-23T18:34:01.219991+0000","last_became_peered":"2024-04-23T18:34:01.219991+0000","last_unstale":"2024-04-23T18:34:55.255404+0000","last_undegraded":"2024-04-23T18:34:55.255404+0000","last_fullsized":"2024-04-23T18:34:55.255404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:29:00.662700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.257995+0000","last_change":"2024-04-23T18:34:00.229625+0000","last_active":"2024-04-23T18:34:55.257995+0000","last_peered":"2024-04-23T18:34:55.257995+0000","last_clean":"2024-04-23T18:34:55.257995+0000","last_became_active":"2024-04-23T18:34:00.229204+0000","last_became_peered":"2024-04-23T18:34:00.229204+0000","last_unstale":"2024-04-23T18:34:55.257995+0000","last_undegraded":"2024-04-23T18:34:55.257995+0000","last_fullsized":"2024-04-23T18:34:55.257995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:42.370870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256319+0000","last_change":"2024-04-23T18:34:00.236000+0000","last_active":"2024-04-23T18:34:55.256319+0000","last_peered":"2024-04-23T18:34:55.256319+0000","last_clean":"2024-04-23T18:34:55.256319+0000","last_became_active":"2024-04-23T18:34:00.235561+0000","last_became_peered":"2024-04-23T18:34:00.235561+0000","last_unstale":"2024-04-23T18:34:55.256319+0000","last_undegraded":"2024-04-23T18:34:55.256319+0000","last_fullsized":"2024-04-23T18:34:55.256319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:07.565489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256199+0000","last_change":"2024-04-23T18:34:01.227955+0000","last_active":"2024-04-23T18:34:55.256199+0000","last_peered":"2024-04-23T18:34:55.256199+0000","last_clean":"2024-04-23T18:34:55.256199+0000","last_became_active":"2024-04-23T18:34:01.225992+0000","last_became_peered":"2024-04-23T18:34:01.225992+0000","last_unstale":"2024-04-23T18:34:55.256199+0000","last_undegraded":"2024-04-23T18:34:55.256199+0000","last_fullsized":"2024-04-23T18:34:55.256199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:05:55.147008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255849+0000","last_change":"2024-04-23T18:34:00.230133+0000","last_active":"2024-04-23T18:34:55.255849+0000","last_peered":"2024-04-23T18:34:55.255849+0000","last_clean":"2024-04-23T18:34:55.255849+0000","last_became_active":"2024-04-23T18:34:00.229596+0000","last_became_peered":"2024-04-23T18:34:00.229596+0000","last_unstale":"2024-04-23T18:34:55.255849+0000","last_undegraded":"2024-04-23T18:34:55.255849+0000","last_fullsized":"2024-04-23T18:34:55.255849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:23:06.064313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286678+0000","last_change":"2024-04-23T18:34:01.229066+0000","last_active":"2024-04-23T18:34:55.286678+0000","last_peered":"2024-04-23T18:34:55.286678+0000","last_clean":"2024-04-23T18:34:55.286678+0000","last_became_active":"2024-04-23T18:34:01.228711+0000","last_became_peered":"2024-04-23T18:34:01.228711+0000","last_unstale":"2024-04-23T18:34:55.286678+0000","last_undegraded":"2024-04-23T18:34:55.286678+0000","last_fullsized":"2024-04-23T18:34:55.286678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:58:30.601270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.287316+0000","last_change":"2024-04-23T18:34:00.231227+0000","last_active":"2024-04-23T18:34:55.287316+0000","last_peered":"2024-04-23T18:34:55.287316+0000","last_clean":"2024-04-23T18:34:55.287316+0000","last_became_active":"2024-04-23T18:34:00.230843+0000","last_became_peered":"2024-04-23T18:34:00.230843+0000","last_unstale":"2024-04-23T18:34:55.287316+0000","last_undegraded":"2024-04-23T18:34:55.287316+0000","last_fullsized":"2024-04-23T18:34:55.287316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:55.416591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256753+0000","last_change":"2024-04-23T18:34:01.233164+0000","last_active":"2024-04-23T18:34:55.256753+0000","last_peered":"2024-04-23T18:34:55.256753+0000","last_clean":"2024-04-23T18:34:55.256753+0000","last_became_active":"2024-04-23T18:34:01.232827+0000","last_became_peered":"2024-04-23T18:34:01.232827+0000","last_unstale":"2024-04-23T18:34:55.256753+0000","last_undegraded":"2024-04-23T18:34:55.256753+0000","last_fullsized":"2024-04-23T18:34:55.256753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:09.635515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286425+0000","last_change":"2024-04-23T18:34:00.225651+0000","last_active":"2024-04-23T18:34:55.286425+0000","last_peered":"2024-04-23T18:34:55.286425+0000","last_clean":"2024-04-23T18:34:55.286425+0000","last_became_active":"2024-04-23T18:34:00.225001+0000","last_became_peered":"2024-04-23T18:34:00.225001+0000","last_unstale":"2024-04-23T18:34:55.286425+0000","last_undegraded":"2024-04-23T18:34:55.286425+0000","last_fullsized":"2024-04-23T18:34:55.286425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:27:44.265388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255780+0000","last_change":"2024-04-23T18:34:01.224131+0000","last_active":"2024-04-23T18:34:55.255780+0000","last_peered":"2024-04-23T18:34:55.255780+0000","last_clean":"2024-04-23T18:34:55.255780+0000","last_became_active":"2024-04-23T18:34:01.223742+0000","last_became_peered":"2024-04-23T18:34:01.223742+0000","last_unstale":"2024-04-23T18:34:55.255780+0000","last_undegraded":"2024-04-23T18:34:55.255780+0000","last_fullsized":"2024-04-23T18:34:55.255780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:29:26.430964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286062+0000","last_change":"2024-04-23T18:34:00.235789+0000","last_active":"2024-04-23T18:34:55.286062+0000","last_peered":"2024-04-23T18:34:55.286062+0000","last_clean":"2024-04-23T18:34:55.286062+0000","last_became_active":"2024-04-23T18:34:00.235452+0000","last_became_peered":"2024-04-23T18:34:00.235452+0000","last_unstale":"2024-04-23T18:34:55.286062+0000","last_undegraded":"2024-04-23T18:34:55.286062+0000","last_fullsized":"2024-04-23T18:34:55.286062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:21.090121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.252908+0000","last_change":"2024-04-23T18:34:01.226156+0000","last_active":"2024-04-23T18:34:55.252908+0000","last_peered":"2024-04-23T18:34:55.252908+0000","last_clean":"2024-04-23T18:34:55.252908+0000","last_became_active":"2024-04-23T18:34:01.225817+0000","last_became_peered":"2024-04-23T18:34:01.225817+0000","last_unstale":"2024-04-23T18:34:55.252908+0000","last_undegraded":"2024-04-23T18:34:55.252908+0000","last_fullsized":"2024-04-23T18:34:55.252908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:36.337139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255363+0000","last_change":"2024-04-23T18:34:00.235116+0000","last_active":"2024-04-23T18:34:55.255363+0000","last_peered":"2024-04-23T18:34:55.255363+0000","last_clean":"2024-04-23T18:34:55.255363+0000","last_became_active":"2024-04-23T18:34:00.234521+0000","last_became_peered":"2024-04-23T18:34:00.234521+0000","last_unstale":"2024-04-23T18:34:55.255363+0000","last_undegraded":"2024-04-23T18:34:55.255363+0000","last_fullsized":"2024-04-23T18:34:55.255363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:51.061164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286352+0000","last_change":"2024-04-23T18:34:01.226445+0000","last_active":"2024-04-23T18:34:55.286352+0000","last_peered":"2024-04-23T18:34:55.286352+0000","last_clean":"2024-04-23T18:34:55.286352+0000","last_became_active":"2024-04-23T18:34:01.225875+0000","last_became_peered":"2024-04-23T18:34:01.225875+0000","last_unstale":"2024-04-23T18:34:55.286352+0000","last_undegraded":"2024-04-23T18:34:55.286352+0000","last_fullsized":"2024-04-23T18:34:55.286352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:48:38.712502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255934+0000","last_change":"2024-04-23T18:34:00.231918+0000","last_active":"2024-04-23T18:34:55.255934+0000","last_peered":"2024-04-23T18:34:55.255934+0000","last_clean":"2024-04-23T18:34:55.255934+0000","last_became_active":"2024-04-23T18:34:00.231550+0000","last_became_peered":"2024-04-23T18:34:00.231550+0000","last_unstale":"2024-04-23T18:34:55.255934+0000","last_undegraded":"2024-04-23T18:34:55.255934+0000","last_fullsized":"2024-04-23T18:34:55.255934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:52.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258228+0000","last_change":"2024-04-23T18:34:01.236360+0000","last_active":"2024-04-23T18:34:55.258228+0000","last_peered":"2024-04-23T18:34:55.258228+0000","last_clean":"2024-04-23T18:34:55.258228+0000","last_became_active":"2024-04-23T18:34:01.236056+0000","last_became_peered":"2024-04-23T18:34:01.236056+0000","last_unstale":"2024-04-23T18:34:55.258228+0000","last_undegraded":"2024-04-23T18:34:55.258228+0000","last_fullsized":"2024-04-23T18:34:55.258228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:54.535158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:13.646507+0000","last_change":"2024-04-23T18:34:00.233159+0000","last_active":"2024-04-23T18:35:13.646507+0000","last_peered":"2024-04-23T18:35:13.646507+0000","last_clean":"2024-04-23T18:35:13.646507+0000","last_became_active":"2024-04-23T18:34:00.232728+0000","last_became_peered":"2024-04-23T18:34:00.232728+0000","last_unstale":"2024-04-23T18:35:13.646507+0000","last_undegraded":"2024-04-23T18:35:13.646507+0000","last_fullsized":"2024-04-23T18:35:13.646507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:23.498057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258882+0000","last_change":"2024-04-23T18:34:01.226134+0000","last_active":"2024-04-23T18:34:55.258882+0000","last_peered":"2024-04-23T18:34:55.258882+0000","last_clean":"2024-04-23T18:34:55.258882+0000","last_became_active":"2024-04-23T18:34:01.225759+0000","last_became_peered":"2024-04-23T18:34:01.225759+0000","last_unstale":"2024-04-23T18:34:55.258882+0000","last_undegraded":"2024-04-23T18:34:55.258882+0000","last_fullsized":"2024-04-23T18:34:55.258882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:24.770595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260552+0000","last_change":"2024-04-23T18:34:00.230054+0000","last_active":"2024-04-23T18:34:55.260552+0000","last_peered":"2024-04-23T18:34:55.260552+0000","last_clean":"2024-04-23T18:34:55.260552+0000","last_became_active":"2024-04-23T18:34:00.229621+0000","last_became_peered":"2024-04-23T18:34:00.229621+0000","last_unstale":"2024-04-23T18:34:55.260552+0000","last_undegraded":"2024-04-23T18:34:55.260552+0000","last_fullsized":"2024-04-23T18:34:55.260552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:13:59.317282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.259118+0000","last_change":"2024-04-23T18:34:01.240053+0000","last_active":"2024-04-23T18:34:55.259118+0000","last_peered":"2024-04-23T18:34:55.259118+0000","last_clean":"2024-04-23T18:34:55.259118+0000","last_became_active":"2024-04-23T18:34:01.239716+0000","last_became_peered":"2024-04-23T18:34:01.239716+0000","last_unstale":"2024-04-23T18:34:55.259118+0000","last_undegraded":"2024-04-23T18:34:55.259118+0000","last_fullsized":"2024-04-23T18:34:55.259118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:54.874374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256530+0000","last_change":"2024-04-23T18:34:00.227914+0000","last_active":"2024-04-23T18:34:55.256530+0000","last_peered":"2024-04-23T18:34:55.256530+0000","last_clean":"2024-04-23T18:34:55.256530+0000","last_became_active":"2024-04-23T18:34:00.227540+0000","last_became_peered":"2024-04-23T18:34:00.227540+0000","last_unstale":"2024-04-23T18:34:55.256530+0000","last_undegraded":"2024-04-23T18:34:55.256530+0000","last_fullsized":"2024-04-23T18:34:55.256530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:18.442397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258982+0000","last_change":"2024-04-23T18:34:01.227319+0000","last_active":"2024-04-23T18:34:55.258982+0000","last_peered":"2024-04-23T18:34:55.258982+0000","last_clean":"2024-04-23T18:34:55.258982+0000","last_became_active":"2024-04-23T18:34:01.226969+0000","last_became_peered":"2024-04-23T18:34:01.226969+0000","last_unstale":"2024-04-23T18:34:55.258982+0000","last_undegraded":"2024-04-23T18:34:55.258982+0000","last_fullsized":"2024-04-23T18:34:55.258982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:19.302388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253082+0000","last_change":"2024-04-23T18:34:01.234466+0000","last_active":"2024-04-23T18:34:55.253082+0000","last_peered":"2024-04-23T18:34:55.253082+0000","last_clean":"2024-04-23T18:34:55.253082+0000","last_became_active":"2024-04-23T18:34:01.234127+0000","last_became_peered":"2024-04-23T18:34:01.234127+0000","last_unstale":"2024-04-23T18:34:55.253082+0000","last_undegraded":"2024-04-23T18:34:55.253082+0000","last_fullsized":"2024-04-23T18:34:55.253082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:51:51.912320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255255+0000","last_change":"2024-04-23T18:34:01.229707+0000","last_active":"2024-04-23T18:34:55.255255+0000","last_peered":"2024-04-23T18:34:55.255255+0000","last_clean":"2024-04-23T18:34:55.255255+0000","last_became_active":"2024-04-23T18:34:01.229311+0000","last_became_peered":"2024-04-23T18:34:01.229311+0000","last_unstale":"2024-04-23T18:34:55.255255+0000","last_undegraded":"2024-04-23T18:34:55.255255+0000","last_fullsized":"2024-04-23T18:34:55.255255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:15:51.972436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255233+0000","last_change":"2024-04-23T18:34:01.227285+0000","last_active":"2024-04-23T18:34:55.255233+0000","last_peered":"2024-04-23T18:34:55.255233+0000","last_clean":"2024-04-23T18:34:55.255233+0000","last_became_active":"2024-04-23T18:34:01.226960+0000","last_became_peered":"2024-04-23T18:34:01.226960+0000","last_unstale":"2024-04-23T18:34:55.255233+0000","last_undegraded":"2024-04-23T18:34:55.255233+0000","last_fullsized":"2024-04-23T18:34:55.255233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:50.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258290+0000","last_change":"2024-04-23T18:34:01.230697+0000","last_active":"2024-04-23T18:34:55.258290+0000","last_peered":"2024-04-23T18:34:55.258290+0000","last_clean":"2024-04-23T18:34:55.258290+0000","last_became_active":"2024-04-23T18:34:01.230352+0000","last_became_peered":"2024-04-23T18:34:01.230352+0000","last_unstale":"2024-04-23T18:34:55.258290+0000","last_undegraded":"2024-04-23T18:34:55.258290+0000","last_fullsized":"2024-04-23T18:34:55.258290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:35:10.609773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256294+0000","last_change":"2024-04-23T18:34:01.226970+0000","last_active":"2024-04-23T18:34:55.256294+0000","last_peered":"2024-04-23T18:34:55.256294+0000","last_clean":"2024-04-23T18:34:55.256294+0000","last_became_active":"2024-04-23T18:34:01.224151+0000","last_became_peered":"2024-04-23T18:34:01.224151+0000","last_unstale":"2024-04-23T18:34:55.256294+0000","last_undegraded":"2024-04-23T18:34:55.256294+0000","last_fullsized":"2024-04-23T18:34:55.256294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:36:15.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253136+0000","last_change":"2024-04-23T18:34:01.234194+0000","last_active":"2024-04-23T18:34:55.253136+0000","last_peered":"2024-04-23T18:34:55.253136+0000","last_clean":"2024-04-23T18:34:55.253136+0000","last_became_active":"2024-04-23T18:34:01.232404+0000","last_became_peered":"2024-04-23T18:34:01.232404+0000","last_unstale":"2024-04-23T18:34:55.253136+0000","last_undegraded":"2024-04-23T18:34:55.253136+0000","last_fullsized":"2024-04-23T18:34:55.253136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:30.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255670+0000","last_change":"2024-04-23T18:34:01.219507+0000","last_active":"2024-04-23T18:34:55.255670+0000","last_peered":"2024-04-23T18:34:55.255670+0000","last_clean":"2024-04-23T18:34:55.255670+0000","last_became_active":"2024-04-23T18:34:01.218997+0000","last_became_peered":"2024-04-23T18:34:01.218997+0000","last_unstale":"2024-04-23T18:34:55.255670+0000","last_undegraded":"2024-04-23T18:34:55.255670+0000","last_fullsized":"2024-04-23T18:34:55.255670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:38.233652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286591+0000","last_change":"2024-04-23T18:34:01.231846+0000","last_active":"2024-04-23T18:34:55.286591+0000","last_peered":"2024-04-23T18:34:55.286591+0000","last_clean":"2024-04-23T18:34:55.286591+0000","last_became_active":"2024-04-23T18:34:01.231467+0000","last_became_peered":"2024-04-23T18:34:01.231467+0000","last_unstale":"2024-04-23T18:34:55.286591+0000","last_undegraded":"2024-04-23T18:34:55.286591+0000","last_fullsized":"2024-04-23T18:34:55.286591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:37:27.891180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260623+0000","last_change":"2024-04-23T18:34:01.237086+0000","last_active":"2024-04-23T18:34:55.260623+0000","last_peered":"2024-04-23T18:34:55.260623+0000","last_clean":"2024-04-23T18:34:55.260623+0000","last_became_active":"2024-04-23T18:34:01.236610+0000","last_became_peered":"2024-04-23T18:34:01.236610+0000","last_unstale":"2024-04-23T18:34:55.260623+0000","last_undegraded":"2024-04-23T18:34:55.260623+0000","last_fullsized":"2024-04-23T18:34:55.260623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:24.916553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286027+0000","last_change":"2024-04-23T18:34:01.231091+0000","last_active":"2024-04-23T18:34:55.286027+0000","last_peered":"2024-04-23T18:34:55.286027+0000","last_clean":"2024-04-23T18:34:55.286027+0000","last_became_active":"2024-04-23T18:34:01.230731+0000","last_became_peered":"2024-04-23T18:34:01.230731+0000","last_unstale":"2024-04-23T18:34:55.286027+0000","last_undegraded":"2024-04-23T18:34:55.286027+0000","last_fullsized":"2024-04-23T18:34:55.286027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:22.771515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286699+0000","last_change":"2024-04-23T18:34:01.231820+0000","last_active":"2024-04-23T18:34:55.286699+0000","last_peered":"2024-04-23T18:34:55.286699+0000","last_clean":"2024-04-23T18:34:55.286699+0000","last_became_active":"2024-04-23T18:34:01.230320+0000","last_became_peered":"2024-04-23T18:34:01.230320+0000","last_unstale":"2024-04-23T18:34:55.286699+0000","last_undegraded":"2024-04-23T18:34:55.286699+0000","last_fullsized":"2024-04-23T18:34:55.286699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:27:35.324323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255907+0000","last_change":"2024-04-23T18:34:01.228723+0000","last_active":"2024-04-23T18:34:55.255907+0000","last_peered":"2024-04-23T18:34:55.255907+0000","last_clean":"2024-04-23T18:34:55.255907+0000","last_became_active":"2024-04-23T18:34:01.227500+0000","last_became_peered":"2024-04-23T18:34:01.227500+0000","last_unstale":"2024-04-23T18:34:55.255907+0000","last_undegraded":"2024-04-23T18:34:55.255907+0000","last_fullsized":"2024-04-23T18:34:55.255907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:17.908021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.252992+0000","last_change":"2024-04-23T18:34:01.227541+0000","last_active":"2024-04-23T18:34:55.252992+0000","last_peered":"2024-04-23T18:34:55.252992+0000","last_clean":"2024-04-23T18:34:55.252992+0000","last_became_active":"2024-04-23T18:34:01.227201+0000","last_became_peered":"2024-04-23T18:34:01.227201+0000","last_unstale":"2024-04-23T18:34:55.252992+0000","last_undegraded":"2024-04-23T18:34:55.252992+0000","last_fullsized":"2024-04-23T18:34:55.252992+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:25:24.886850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282171,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66669,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":730,"num_read_kb":437,"num_write":236,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":200007,"data_compressed":5622,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":462787,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":1112778,"data_compressed":43551,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":462214,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":462480,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1055110,"data_compressed":41677,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":461931,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":522309,"data_compressed":7665,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":463661,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":523311,"data_compressed":7665,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":461756,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":461792,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":1054788,"data_compressed":41677,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34247,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34247,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34247,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61217,"data_compressed":1874,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62219,"data_compressed":1874,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61201,"data_compressed":1874,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:35:19.768 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T18:35:20.004 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:20.398 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:20 smithi132 ceph-mon[29178]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:35:20.399 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:35:20.399 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:35:20.399 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:20 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:20.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:20 smithi148 ceph-mon[32311]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:35:20.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:35:20.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:35:20.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:20 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:20.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:20 smithi157 ceph-mon[32580]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:35:20.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:35:20.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:35:20.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:20 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' 2024-04-23T18:35:21.442 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:21 smithi132 ceph-mon[29178]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:35:21.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:21 smithi148 ceph-mon[32311]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:35:21.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:21 smithi157 ceph-mon[32580]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:35:21.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.724+0000 7fda66ba0700 1 -- 172.21.15.132:0/1529663761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda601064e0 msgr2=0x7fda6010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.724+0000 7fda66ba0700 1 --2- 172.21.15.132:0/1529663761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda601064e0 0x7fda6010e020 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fda5800b0a0 tx=0x7fda5800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.724+0000 7fda66ba0700 1 -- 172.21.15.132:0/1529663761 shutdown_connections 2024-04-23T18:35:21.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.724+0000 7fda66ba0700 1 --2- 172.21.15.132:0/1529663761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda601064e0 0x7fda6010e020 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.724+0000 7fda66ba0700 1 --2- 172.21.15.132:0/1529663761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda60101640 0x7fda60101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.724+0000 7fda66ba0700 1 --2- 172.21.15.132:0/1529663761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda60101ff0 0x7fda60105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.724+0000 7fda66ba0700 1 -- 172.21.15.132:0/1529663761 >> 172.21.15.132:0/1529663761 conn(0x7fda60076c60 msgr2=0x7fda60077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:21.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.724+0000 7fda66ba0700 1 -- 172.21.15.132:0/1529663761 shutdown_connections 2024-04-23T18:35:21.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.724+0000 7fda66ba0700 1 -- 172.21.15.132:0/1529663761 wait complete. 2024-04-23T18:35:21.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.724+0000 7fda66ba0700 1 Processor -- start 2024-04-23T18:35:21.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.724+0000 7fda66ba0700 1 -- start start 2024-04-23T18:35:21.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda66ba0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda60101640 0x7fda60199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda66ba0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60101ff0 0x7fda60199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda66ba0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda601064e0 0x7fda6019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda66ba0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fda60111bc0 con 0x7fda60101ff0 2024-04-23T18:35:21.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda66ba0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda60111a40 con 0x7fda60101640 2024-04-23T18:35:21.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda66ba0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda60111d40 con 0x7fda601064e0 2024-04-23T18:35:21.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60101ff0 0x7fda60199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60101ff0 0x7fda60199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49052/0 (socket says 172.21.15.132:49052) 2024-04-23T18:35:21.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda5ffff700 1 -- 172.21.15.132:0/863281546 learned_addr learned my addr 172.21.15.132:0/863281546 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:21.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda6493c700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda60101640 0x7fda60199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda6513d700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda601064e0 0x7fda6019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda5ffff700 1 -- 172.21.15.132:0/863281546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda601064e0 msgr2=0x7fda6019d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda5ffff700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda601064e0 0x7fda6019d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda5ffff700 1 -- 172.21.15.132:0/863281546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda60101640 msgr2=0x7fda60199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda5ffff700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda60101640 0x7fda60199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda5ffff700 1 -- 172.21.15.132:0/863281546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6019e030 con 0x7fda60101ff0 2024-04-23T18:35:21.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda5ffff700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60101ff0 0x7fda60199560 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00c9b0 tx=0x7fda4c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:21.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda5dffb700 1 -- 172.21.15.132:0/863281546 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c007aa0 con 0x7fda60101ff0 2024-04-23T18:35:21.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.725+0000 7fda66ba0700 1 -- 172.21.15.132:0/863281546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6019e320 con 0x7fda60101ff0 2024-04-23T18:35:21.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.726+0000 7fda5dffb700 1 -- 172.21.15.132:0/863281546 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda4c007c40 con 0x7fda60101ff0 2024-04-23T18:35:21.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.726+0000 7fda5dffb700 1 -- 172.21.15.132:0/863281546 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c018660 con 0x7fda60101ff0 2024-04-23T18:35:21.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.726+0000 7fda66ba0700 1 -- 172.21.15.132:0/863281546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda601a5c20 con 0x7fda60101ff0 2024-04-23T18:35:21.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.727+0000 7fda66ba0700 1 -- 172.21.15.132:0/863281546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda60069730 con 0x7fda60101ff0 2024-04-23T18:35:21.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.728+0000 7fda5dffb700 1 -- 172.21.15.132:0/863281546 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fda4c00f450 con 0x7fda60101ff0 2024-04-23T18:35:21.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.728+0000 7fda5dffb700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fda500a3d50 0x7fda500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.728+0000 7fda6493c700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fda500a3d50 0x7fda500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.728+0000 7fda5dffb700 1 -- 172.21.15.132:0/863281546 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fda4c0d03e0 con 0x7fda60101ff0 2024-04-23T18:35:21.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.729+0000 7fda6493c700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fda500a3d50 0x7fda500a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda54009a10 tx=0x7fda54009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:21.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.733+0000 7fda5dffb700 1 -- 172.21.15.132:0/863281546 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda4c014030 con 0x7fda60101ff0 2024-04-23T18:35:21.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.926+0000 7fda66ba0700 1 -- 172.21.15.132:0/863281546 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda60062e80 con 0x7fda500a3d50 2024-04-23T18:35:21.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.933+0000 7fda5dffb700 1 -- 172.21.15.132:0/863281546 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259065 (secure 0 0 0) 0x7fda60062e80 con 0x7fda500a3d50 2024-04-23T18:35:21.928 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:21.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.936+0000 7fda66ba0700 1 -- 172.21.15.132:0/863281546 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fda500a3d50 msgr2=0x7fda500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.936+0000 7fda66ba0700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fda500a3d50 0x7fda500a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda54009a10 tx=0x7fda54009190 comp rx=0 tx=0).stop 2024-04-23T18:35:21.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.936+0000 7fda66ba0700 1 -- 172.21.15.132:0/863281546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60101ff0 msgr2=0x7fda60199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.936+0000 7fda66ba0700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60101ff0 0x7fda60199560 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00c9b0 tx=0x7fda4c00ccc0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.936+0000 7fda66ba0700 1 -- 172.21.15.132:0/863281546 shutdown_connections 2024-04-23T18:35:21.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.936+0000 7fda66ba0700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda601064e0 0x7fda6019d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.936+0000 7fda66ba0700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda60101640 0x7fda60199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.936+0000 7fda66ba0700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fda500a3d50 0x7fda500a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.936+0000 7fda66ba0700 1 --2- 172.21.15.132:0/863281546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60101ff0 0x7fda60199560 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.936+0000 7fda66ba0700 1 -- 172.21.15.132:0/863281546 >> 172.21.15.132:0/863281546 conn(0x7fda60076c60 msgr2=0x7fda6010eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:21.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.936+0000 7fda66ba0700 1 -- 172.21.15.132:0/863281546 shutdown_connections 2024-04-23T18:35:21.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:21.936+0000 7fda66ba0700 1 -- 172.21.15.132:0/863281546 wait complete. 2024-04-23T18:35:21.931 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-04-23T18:35:22.614 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-04-23T18:35:20.146261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657271,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":876,"num_read_kb":561,"num_write":419,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352752,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589072,"statfs":{"total":1151940427776,"available":1151579209728,"internally_reserved":0,"allocated":12128256,"data_stored":7504917,"data_compressed":182772,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003308"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253272+0000","last_change":"2024-04-23T18:34:01.231204+0000","last_active":"2024-04-23T18:34:55.253272+0000","last_peered":"2024-04-23T18:34:55.253272+0000","last_clean":"2024-04-23T18:34:55.253272+0000","last_became_active":"2024-04-23T18:34:01.230836+0000","last_became_peered":"2024-04-23T18:34:01.230836+0000","last_unstale":"2024-04-23T18:34:55.253272+0000","last_undegraded":"2024-04-23T18:34:55.253272+0000","last_fullsized":"2024-04-23T18:34:55.253272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:57.744807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256840+0000","last_change":"2024-04-23T18:34:01.225833+0000","last_active":"2024-04-23T18:34:55.256840+0000","last_peered":"2024-04-23T18:34:55.256840+0000","last_clean":"2024-04-23T18:34:55.256840+0000","last_became_active":"2024-04-23T18:34:01.225098+0000","last_became_peered":"2024-04-23T18:34:01.225098+0000","last_unstale":"2024-04-23T18:34:55.256840+0000","last_undegraded":"2024-04-23T18:34:55.256840+0000","last_fullsized":"2024-04-23T18:34:55.256840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:20.720137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256596+0000","last_change":"2024-04-23T18:34:01.235269+0000","last_active":"2024-04-23T18:34:55.256596+0000","last_peered":"2024-04-23T18:34:55.256596+0000","last_clean":"2024-04-23T18:34:55.256596+0000","last_became_active":"2024-04-23T18:34:01.234921+0000","last_became_peered":"2024-04-23T18:34:01.234921+0000","last_unstale":"2024-04-23T18:34:55.256596+0000","last_undegraded":"2024-04-23T18:34:55.256596+0000","last_fullsized":"2024-04-23T18:34:55.256596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:06:20.645693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.287914+0000","last_change":"2024-04-23T18:34:01.233316+0000","last_active":"2024-04-23T18:34:55.287914+0000","last_peered":"2024-04-23T18:34:55.287914+0000","last_clean":"2024-04-23T18:34:55.287914+0000","last_became_active":"2024-04-23T18:34:01.231636+0000","last_became_peered":"2024-04-23T18:34:01.231636+0000","last_unstale":"2024-04-23T18:34:55.287914+0000","last_undegraded":"2024-04-23T18:34:55.287914+0000","last_fullsized":"2024-04-23T18:34:55.287914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:15.837129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255885+0000","last_change":"2024-04-23T18:34:01.228073+0000","last_active":"2024-04-23T18:34:55.255885+0000","last_peered":"2024-04-23T18:34:55.255885+0000","last_clean":"2024-04-23T18:34:55.255885+0000","last_became_active":"2024-04-23T18:34:01.227701+0000","last_became_peered":"2024-04-23T18:34:01.227701+0000","last_unstale":"2024-04-23T18:34:55.255885+0000","last_undegraded":"2024-04-23T18:34:55.255885+0000","last_fullsized":"2024-04-23T18:34:55.255885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:17:26.556498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258233+0000","last_change":"2024-04-23T18:34:55.258233+0000","last_active":"2024-04-23T18:34:55.258233+0000","last_peered":"2024-04-23T18:34:55.258233+0000","last_clean":"2024-04-23T18:34:55.258233+0000","last_became_active":"2024-04-23T18:34:55.257836+0000","last_became_peered":"2024-04-23T18:34:55.257836+0000","last_unstale":"2024-04-23T18:34:55.258233+0000","last_undegraded":"2024-04-23T18:34:55.258233+0000","last_fullsized":"2024-04-23T18:34:55.258233+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:20.279369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260737+0000","last_change":"2024-04-23T18:34:55.260737+0000","last_active":"2024-04-23T18:34:55.260737+0000","last_peered":"2024-04-23T18:34:55.260737+0000","last_clean":"2024-04-23T18:34:55.260737+0000","last_became_active":"2024-04-23T18:34:55.260231+0000","last_became_peered":"2024-04-23T18:34:55.260231+0000","last_unstale":"2024-04-23T18:34:55.260737+0000","last_undegraded":"2024-04-23T18:34:55.260737+0000","last_fullsized":"2024-04-23T18:34:55.260737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:45.292690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.259963+0000","last_change":"2024-04-23T18:34:55.259963+0000","last_active":"2024-04-23T18:34:55.259963+0000","last_peered":"2024-04-23T18:34:55.259963+0000","last_clean":"2024-04-23T18:34:55.259963+0000","last_became_active":"2024-04-23T18:34:55.259342+0000","last_became_peered":"2024-04-23T18:34:55.259342+0000","last_unstale":"2024-04-23T18:34:55.259963+0000","last_undegraded":"2024-04-23T18:34:55.259963+0000","last_fullsized":"2024-04-23T18:34:55.259963+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:42:36.612809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286947+0000","last_change":"2024-04-23T18:34:01.234094+0000","last_active":"2024-04-23T18:34:55.286947+0000","last_peered":"2024-04-23T18:34:55.286947+0000","last_clean":"2024-04-23T18:34:55.286947+0000","last_became_active":"2024-04-23T18:34:01.232499+0000","last_became_peered":"2024-04-23T18:34:01.232499+0000","last_unstale":"2024-04-23T18:34:55.286947+0000","last_undegraded":"2024-04-23T18:34:55.286947+0000","last_fullsized":"2024-04-23T18:34:55.286947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:33.004245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253537+0000","last_change":"2024-04-23T18:34:01.229175+0000","last_active":"2024-04-23T18:34:55.253537+0000","last_peered":"2024-04-23T18:34:55.253537+0000","last_clean":"2024-04-23T18:34:55.253537+0000","last_became_active":"2024-04-23T18:34:01.228837+0000","last_became_peered":"2024-04-23T18:34:01.228837+0000","last_unstale":"2024-04-23T18:34:55.253537+0000","last_undegraded":"2024-04-23T18:34:55.253537+0000","last_fullsized":"2024-04-23T18:34:55.253537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:57.625110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258853+0000","last_change":"2024-04-23T18:34:01.235557+0000","last_active":"2024-04-23T18:34:55.258853+0000","last_peered":"2024-04-23T18:34:55.258853+0000","last_clean":"2024-04-23T18:34:55.258853+0000","last_became_active":"2024-04-23T18:34:01.235184+0000","last_became_peered":"2024-04-23T18:34:01.235184+0000","last_unstale":"2024-04-23T18:34:55.258853+0000","last_undegraded":"2024-04-23T18:34:55.258853+0000","last_fullsized":"2024-04-23T18:34:55.258853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:37:27.138041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.287745+0000","last_change":"2024-04-23T18:34:01.234070+0000","last_active":"2024-04-23T18:34:55.287745+0000","last_peered":"2024-04-23T18:34:55.287745+0000","last_clean":"2024-04-23T18:34:55.287745+0000","last_became_active":"2024-04-23T18:34:01.233727+0000","last_became_peered":"2024-04-23T18:34:01.233727+0000","last_unstale":"2024-04-23T18:34:55.287745+0000","last_undegraded":"2024-04-23T18:34:55.287745+0000","last_fullsized":"2024-04-23T18:34:55.287745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:59:16.489819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256003+0000","last_change":"2024-04-23T18:34:01.228256+0000","last_active":"2024-04-23T18:34:55.256003+0000","last_peered":"2024-04-23T18:34:55.256003+0000","last_clean":"2024-04-23T18:34:55.256003+0000","last_became_active":"2024-04-23T18:34:01.224400+0000","last_became_peered":"2024-04-23T18:34:01.224400+0000","last_unstale":"2024-04-23T18:34:55.256003+0000","last_undegraded":"2024-04-23T18:34:55.256003+0000","last_fullsized":"2024-04-23T18:34:55.256003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:37:28.401695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256371+0000","last_change":"2024-04-23T18:34:01.227250+0000","last_active":"2024-04-23T18:34:55.256371+0000","last_peered":"2024-04-23T18:34:55.256371+0000","last_clean":"2024-04-23T18:34:55.256371+0000","last_became_active":"2024-04-23T18:34:01.226937+0000","last_became_peered":"2024-04-23T18:34:01.226937+0000","last_unstale":"2024-04-23T18:34:55.256371+0000","last_undegraded":"2024-04-23T18:34:55.256371+0000","last_fullsized":"2024-04-23T18:34:55.256371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:26:20.801691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256119+0000","last_change":"2024-04-23T18:34:01.232351+0000","last_active":"2024-04-23T18:34:55.256119+0000","last_peered":"2024-04-23T18:34:55.256119+0000","last_clean":"2024-04-23T18:34:55.256119+0000","last_became_active":"2024-04-23T18:34:01.231987+0000","last_became_peered":"2024-04-23T18:34:01.231987+0000","last_unstale":"2024-04-23T18:34:55.256119+0000","last_undegraded":"2024-04-23T18:34:55.256119+0000","last_fullsized":"2024-04-23T18:34:55.256119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:50.886496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286235+0000","last_change":"2024-04-23T18:34:01.232196+0000","last_active":"2024-04-23T18:34:55.286235+0000","last_peered":"2024-04-23T18:34:55.286235+0000","last_clean":"2024-04-23T18:34:55.286235+0000","last_became_active":"2024-04-23T18:34:01.231746+0000","last_became_peered":"2024-04-23T18:34:01.231746+0000","last_unstale":"2024-04-23T18:34:55.286235+0000","last_undegraded":"2024-04-23T18:34:55.286235+0000","last_fullsized":"2024-04-23T18:34:55.286235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:58.888549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253346+0000","last_change":"2024-04-23T18:34:01.231509+0000","last_active":"2024-04-23T18:34:55.253346+0000","last_peered":"2024-04-23T18:34:55.253346+0000","last_clean":"2024-04-23T18:34:55.253346+0000","last_became_active":"2024-04-23T18:34:01.231017+0000","last_became_peered":"2024-04-23T18:34:01.231017+0000","last_unstale":"2024-04-23T18:34:55.253346+0000","last_undegraded":"2024-04-23T18:34:55.253346+0000","last_fullsized":"2024-04-23T18:34:55.253346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:04:51.929462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255975+0000","last_change":"2024-04-23T18:34:01.223285+0000","last_active":"2024-04-23T18:34:55.255975+0000","last_peered":"2024-04-23T18:34:55.255975+0000","last_clean":"2024-04-23T18:34:55.255975+0000","last_became_active":"2024-04-23T18:34:01.222962+0000","last_became_peered":"2024-04-23T18:34:01.222962+0000","last_unstale":"2024-04-23T18:34:55.255975+0000","last_undegraded":"2024-04-23T18:34:55.255975+0000","last_fullsized":"2024-04-23T18:34:55.255975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:18.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255490+0000","last_change":"2024-04-23T18:34:01.225922+0000","last_active":"2024-04-23T18:34:55.255490+0000","last_peered":"2024-04-23T18:34:55.255490+0000","last_clean":"2024-04-23T18:34:55.255490+0000","last_became_active":"2024-04-23T18:34:01.225235+0000","last_became_peered":"2024-04-23T18:34:01.225235+0000","last_unstale":"2024-04-23T18:34:55.255490+0000","last_undegraded":"2024-04-23T18:34:55.255490+0000","last_fullsized":"2024-04-23T18:34:55.255490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:33.480782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:17.636412+0000","last_change":"2024-04-23T18:34:00.224606+0000","last_active":"2024-04-23T18:35:17.636412+0000","last_peered":"2024-04-23T18:35:17.636412+0000","last_clean":"2024-04-23T18:35:17.636412+0000","last_became_active":"2024-04-23T18:34:00.224291+0000","last_became_peered":"2024-04-23T18:34:00.224291+0000","last_unstale":"2024-04-23T18:35:17.636412+0000","last_undegraded":"2024-04-23T18:35:17.636412+0000","last_fullsized":"2024-04-23T18:35:17.636412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:40.734704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286232+0000","last_change":"2024-04-23T18:34:01.235023+0000","last_active":"2024-04-23T18:34:55.286232+0000","last_peered":"2024-04-23T18:34:55.286232+0000","last_clean":"2024-04-23T18:34:55.286232+0000","last_became_active":"2024-04-23T18:34:01.234672+0000","last_became_peered":"2024-04-23T18:34:01.234672+0000","last_unstale":"2024-04-23T18:34:55.286232+0000","last_undegraded":"2024-04-23T18:34:55.286232+0000","last_fullsized":"2024-04-23T18:34:55.286232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:33.734343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286517+0000","last_change":"2024-04-23T18:34:00.232552+0000","last_active":"2024-04-23T18:34:55.286517+0000","last_peered":"2024-04-23T18:34:55.286517+0000","last_clean":"2024-04-23T18:34:55.286517+0000","last_became_active":"2024-04-23T18:34:00.232226+0000","last_became_peered":"2024-04-23T18:34:00.232226+0000","last_unstale":"2024-04-23T18:34:55.286517+0000","last_undegraded":"2024-04-23T18:34:55.286517+0000","last_fullsized":"2024-04-23T18:34:55.286517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.563757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.257141+0000","last_change":"2024-04-23T18:34:01.236013+0000","last_active":"2024-04-23T18:34:55.257141+0000","last_peered":"2024-04-23T18:34:55.257141+0000","last_clean":"2024-04-23T18:34:55.257141+0000","last_became_active":"2024-04-23T18:34:01.235590+0000","last_became_peered":"2024-04-23T18:34:01.235590+0000","last_unstale":"2024-04-23T18:34:55.257141+0000","last_undegraded":"2024-04-23T18:34:55.257141+0000","last_fullsized":"2024-04-23T18:34:55.257141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:45.741648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.298010+0000","last_change":"2024-04-23T18:34:55.298010+0000","last_active":"2024-04-23T18:34:55.298010+0000","last_peered":"2024-04-23T18:34:55.298010+0000","last_clean":"2024-04-23T18:34:55.298010+0000","last_became_active":"2024-04-23T18:34:55.288221+0000","last_became_peered":"2024-04-23T18:34:55.288221+0000","last_unstale":"2024-04-23T18:34:55.298010+0000","last_undegraded":"2024-04-23T18:34:55.298010+0000","last_fullsized":"2024-04-23T18:34:55.298010+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:21:35.469676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286457+0000","last_change":"2024-04-23T18:34:01.230435+0000","last_active":"2024-04-23T18:34:55.286457+0000","last_peered":"2024-04-23T18:34:55.286457+0000","last_clean":"2024-04-23T18:34:55.286457+0000","last_became_active":"2024-04-23T18:34:01.230095+0000","last_became_peered":"2024-04-23T18:34:01.230095+0000","last_unstale":"2024-04-23T18:34:55.286457+0000","last_undegraded":"2024-04-23T18:34:55.286457+0000","last_fullsized":"2024-04-23T18:34:55.286457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:31.653254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256171+0000","last_change":"2024-04-23T18:34:00.227853+0000","last_active":"2024-04-23T18:34:55.256171+0000","last_peered":"2024-04-23T18:34:55.256171+0000","last_clean":"2024-04-23T18:34:55.256171+0000","last_became_active":"2024-04-23T18:34:00.226596+0000","last_became_peered":"2024-04-23T18:34:00.226596+0000","last_unstale":"2024-04-23T18:34:55.256171+0000","last_undegraded":"2024-04-23T18:34:55.256171+0000","last_fullsized":"2024-04-23T18:34:55.256171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:16:10.077969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260701+0000","last_change":"2024-04-23T18:34:01.235955+0000","last_active":"2024-04-23T18:34:55.260701+0000","last_peered":"2024-04-23T18:34:55.260701+0000","last_clean":"2024-04-23T18:34:55.260701+0000","last_became_active":"2024-04-23T18:34:01.235609+0000","last_became_peered":"2024-04-23T18:34:01.235609+0000","last_unstale":"2024-04-23T18:34:55.260701+0000","last_undegraded":"2024-04-23T18:34:55.260701+0000","last_fullsized":"2024-04-23T18:34:55.260701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:07:39.480213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":102,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:17.622471+0000","last_change":"2024-04-23T18:34:00.226319+0000","last_active":"2024-04-23T18:35:17.622471+0000","last_peered":"2024-04-23T18:35:17.622471+0000","last_clean":"2024-04-23T18:35:17.622471+0000","last_became_active":"2024-04-23T18:34:00.225796+0000","last_became_peered":"2024-04-23T18:34:00.225796+0000","last_unstale":"2024-04-23T18:35:17.622471+0000","last_undegraded":"2024-04-23T18:35:17.622471+0000","last_fullsized":"2024-04-23T18:35:17.622471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:11:18.309181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255891+0000","last_change":"2024-04-23T18:34:01.227336+0000","last_active":"2024-04-23T18:34:55.255891+0000","last_peered":"2024-04-23T18:34:55.255891+0000","last_clean":"2024-04-23T18:34:55.255891+0000","last_became_active":"2024-04-23T18:34:01.226987+0000","last_became_peered":"2024-04-23T18:34:01.226987+0000","last_unstale":"2024-04-23T18:34:55.255891+0000","last_undegraded":"2024-04-23T18:34:55.255891+0000","last_fullsized":"2024-04-23T18:34:55.255891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:22:30.058673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.294322+0000","last_change":"2024-04-23T18:34:55.294322+0000","last_active":"2024-04-23T18:34:55.294322+0000","last_peered":"2024-04-23T18:34:55.294322+0000","last_clean":"2024-04-23T18:34:55.294322+0000","last_became_active":"2024-04-23T18:34:55.288855+0000","last_became_peered":"2024-04-23T18:34:55.288855+0000","last_unstale":"2024-04-23T18:34:55.294322+0000","last_undegraded":"2024-04-23T18:34:55.294322+0000","last_fullsized":"2024-04-23T18:34:55.294322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:07:50.097696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.287339+0000","last_change":"2024-04-23T18:34:55.287339+0000","last_active":"2024-04-23T18:34:55.287339+0000","last_peered":"2024-04-23T18:34:55.287339+0000","last_clean":"2024-04-23T18:34:55.287339+0000","last_became_active":"2024-04-23T18:34:55.287062+0000","last_became_peered":"2024-04-23T18:34:55.287062+0000","last_unstale":"2024-04-23T18:34:55.287339+0000","last_undegraded":"2024-04-23T18:34:55.287339+0000","last_fullsized":"2024-04-23T18:34:55.287339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:44.864600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255616+0000","last_change":"2024-04-23T18:34:00.227595+0000","last_active":"2024-04-23T18:34:55.255616+0000","last_peered":"2024-04-23T18:34:55.255616+0000","last_clean":"2024-04-23T18:34:55.255616+0000","last_became_active":"2024-04-23T18:34:00.226830+0000","last_became_peered":"2024-04-23T18:34:00.226830+0000","last_unstale":"2024-04-23T18:34:55.255616+0000","last_undegraded":"2024-04-23T18:34:55.255616+0000","last_fullsized":"2024-04-23T18:34:55.255616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:20.661984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286109+0000","last_change":"2024-04-23T18:34:01.225053+0000","last_active":"2024-04-23T18:34:55.286109+0000","last_peered":"2024-04-23T18:34:55.286109+0000","last_clean":"2024-04-23T18:34:55.286109+0000","last_became_active":"2024-04-23T18:34:01.224723+0000","last_became_peered":"2024-04-23T18:34:01.224723+0000","last_unstale":"2024-04-23T18:34:55.286109+0000","last_undegraded":"2024-04-23T18:34:55.286109+0000","last_fullsized":"2024-04-23T18:34:55.286109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:01:51.486691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260492+0000","last_change":"2024-04-23T18:34:00.224783+0000","last_active":"2024-04-23T18:34:55.260492+0000","last_peered":"2024-04-23T18:34:55.260492+0000","last_clean":"2024-04-23T18:34:55.260492+0000","last_became_active":"2024-04-23T18:34:00.224436+0000","last_became_peered":"2024-04-23T18:34:00.224436+0000","last_unstale":"2024-04-23T18:34:55.260492+0000","last_undegraded":"2024-04-23T18:34:55.260492+0000","last_fullsized":"2024-04-23T18:34:55.260492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:33.829188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.254798+0000","last_change":"2024-04-23T18:34:01.224283+0000","last_active":"2024-04-23T18:34:55.254798+0000","last_peered":"2024-04-23T18:34:55.254798+0000","last_clean":"2024-04-23T18:34:55.254798+0000","last_became_active":"2024-04-23T18:34:01.223949+0000","last_became_peered":"2024-04-23T18:34:01.223949+0000","last_unstale":"2024-04-23T18:34:55.254798+0000","last_undegraded":"2024-04-23T18:34:55.254798+0000","last_fullsized":"2024-04-23T18:34:55.254798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:50.560687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":161,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:14.617770+0000","last_change":"2024-04-23T18:34:00.218453+0000","last_active":"2024-04-23T18:35:14.617770+0000","last_peered":"2024-04-23T18:35:14.617770+0000","last_clean":"2024-04-23T18:35:14.617770+0000","last_became_active":"2024-04-23T18:34:00.218036+0000","last_became_peered":"2024-04-23T18:34:00.218036+0000","last_unstale":"2024-04-23T18:35:14.617770+0000","last_undegraded":"2024-04-23T18:35:14.617770+0000","last_fullsized":"2024-04-23T18:35:14.617770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:49.485185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286939+0000","last_change":"2024-04-23T18:34:01.231143+0000","last_active":"2024-04-23T18:34:55.286939+0000","last_peered":"2024-04-23T18:34:55.286939+0000","last_clean":"2024-04-23T18:34:55.286939+0000","last_became_active":"2024-04-23T18:34:01.230812+0000","last_became_peered":"2024-04-23T18:34:01.230812+0000","last_unstale":"2024-04-23T18:34:55.286939+0000","last_undegraded":"2024-04-23T18:34:55.286939+0000","last_fullsized":"2024-04-23T18:34:55.286939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:28.704807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258862+0000","last_change":"2024-04-23T18:34:00.235711+0000","last_active":"2024-04-23T18:34:55.258862+0000","last_peered":"2024-04-23T18:34:55.258862+0000","last_clean":"2024-04-23T18:34:55.258862+0000","last_became_active":"2024-04-23T18:34:00.235333+0000","last_became_peered":"2024-04-23T18:34:00.235333+0000","last_unstale":"2024-04-23T18:34:55.258862+0000","last_undegraded":"2024-04-23T18:34:55.258862+0000","last_fullsized":"2024-04-23T18:34:55.258862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:51.320744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.257055+0000","last_change":"2024-04-23T18:34:01.238291+0000","last_active":"2024-04-23T18:34:55.257055+0000","last_peered":"2024-04-23T18:34:55.257055+0000","last_clean":"2024-04-23T18:34:55.257055+0000","last_became_active":"2024-04-23T18:34:01.237972+0000","last_became_peered":"2024-04-23T18:34:01.237972+0000","last_unstale":"2024-04-23T18:34:55.257055+0000","last_undegraded":"2024-04-23T18:34:55.257055+0000","last_fullsized":"2024-04-23T18:34:55.257055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:36:17.108170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260912+0000","last_change":"2024-04-23T18:34:00.227249+0000","last_active":"2024-04-23T18:34:55.260912+0000","last_peered":"2024-04-23T18:34:55.260912+0000","last_clean":"2024-04-23T18:34:55.260912+0000","last_became_active":"2024-04-23T18:34:00.226854+0000","last_became_peered":"2024-04-23T18:34:00.226854+0000","last_unstale":"2024-04-23T18:34:55.260912+0000","last_undegraded":"2024-04-23T18:34:55.260912+0000","last_fullsized":"2024-04-23T18:34:55.260912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:49.619906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286572+0000","last_change":"2024-04-23T18:34:01.225518+0000","last_active":"2024-04-23T18:34:55.286572+0000","last_peered":"2024-04-23T18:34:55.286572+0000","last_clean":"2024-04-23T18:34:55.286572+0000","last_became_active":"2024-04-23T18:34:01.224907+0000","last_became_peered":"2024-04-23T18:34:01.224907+0000","last_unstale":"2024-04-23T18:34:55.286572+0000","last_undegraded":"2024-04-23T18:34:55.286572+0000","last_fullsized":"2024-04-23T18:34:55.286572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:19.876629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":196,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:14.617719+0000","last_change":"2024-04-23T18:34:00.224770+0000","last_active":"2024-04-23T18:35:14.617719+0000","last_peered":"2024-04-23T18:35:14.617719+0000","last_clean":"2024-04-23T18:35:14.617719+0000","last_became_active":"2024-04-23T18:34:00.224395+0000","last_became_peered":"2024-04-23T18:34:00.224395+0000","last_unstale":"2024-04-23T18:35:14.617719+0000","last_undegraded":"2024-04-23T18:35:14.617719+0000","last_fullsized":"2024-04-23T18:35:14.617719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:14:09.172479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":148,"num_read_kb":132,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256208+0000","last_change":"2024-04-23T18:34:01.227771+0000","last_active":"2024-04-23T18:34:55.256208+0000","last_peered":"2024-04-23T18:34:55.256208+0000","last_clean":"2024-04-23T18:34:55.256208+0000","last_became_active":"2024-04-23T18:34:01.225063+0000","last_became_peered":"2024-04-23T18:34:01.225063+0000","last_unstale":"2024-04-23T18:34:55.256208+0000","last_undegraded":"2024-04-23T18:34:55.256208+0000","last_fullsized":"2024-04-23T18:34:55.256208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:53.921587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255405+0000","last_change":"2024-04-23T18:34:00.230455+0000","last_active":"2024-04-23T18:34:55.255405+0000","last_peered":"2024-04-23T18:34:55.255405+0000","last_clean":"2024-04-23T18:34:55.255405+0000","last_became_active":"2024-04-23T18:34:00.229934+0000","last_became_peered":"2024-04-23T18:34:00.229934+0000","last_unstale":"2024-04-23T18:34:55.255405+0000","last_undegraded":"2024-04-23T18:34:55.255405+0000","last_fullsized":"2024-04-23T18:34:55.255405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:26:34.787606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256945+0000","last_change":"2024-04-23T18:34:01.225757+0000","last_active":"2024-04-23T18:34:55.256945+0000","last_peered":"2024-04-23T18:34:55.256945+0000","last_clean":"2024-04-23T18:34:55.256945+0000","last_became_active":"2024-04-23T18:34:01.225447+0000","last_became_peered":"2024-04-23T18:34:01.225447+0000","last_unstale":"2024-04-23T18:34:55.256945+0000","last_undegraded":"2024-04-23T18:34:55.256945+0000","last_fullsized":"2024-04-23T18:34:55.256945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:17.212135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260447+0000","last_change":"2024-04-23T18:34:00.231749+0000","last_active":"2024-04-23T18:34:55.260447+0000","last_peered":"2024-04-23T18:34:55.260447+0000","last_clean":"2024-04-23T18:34:55.260447+0000","last_became_active":"2024-04-23T18:34:00.231207+0000","last_became_peered":"2024-04-23T18:34:00.231207+0000","last_unstale":"2024-04-23T18:34:55.260447+0000","last_undegraded":"2024-04-23T18:34:55.260447+0000","last_fullsized":"2024-04-23T18:34:55.260447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:23:05.723265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258587+0000","last_change":"2024-04-23T18:34:01.241116+0000","last_active":"2024-04-23T18:34:55.258587+0000","last_peered":"2024-04-23T18:34:55.258587+0000","last_clean":"2024-04-23T18:34:55.258587+0000","last_became_active":"2024-04-23T18:34:01.240785+0000","last_became_peered":"2024-04-23T18:34:01.240785+0000","last_unstale":"2024-04-23T18:34:55.258587+0000","last_undegraded":"2024-04-23T18:34:55.258587+0000","last_fullsized":"2024-04-23T18:34:55.258587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:48.323241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:13.638111+0000","last_change":"2024-04-23T18:34:00.228578+0000","last_active":"2024-04-23T18:35:13.638111+0000","last_peered":"2024-04-23T18:35:13.638111+0000","last_clean":"2024-04-23T18:35:13.638111+0000","last_became_active":"2024-04-23T18:34:00.228238+0000","last_became_peered":"2024-04-23T18:34:00.228238+0000","last_unstale":"2024-04-23T18:35:13.638111+0000","last_undegraded":"2024-04-23T18:35:13.638111+0000","last_fullsized":"2024-04-23T18:35:13.638111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:58:21.283593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.257726+0000","last_change":"2024-04-23T18:34:01.239481+0000","last_active":"2024-04-23T18:34:55.257726+0000","last_peered":"2024-04-23T18:34:55.257726+0000","last_clean":"2024-04-23T18:34:55.257726+0000","last_became_active":"2024-04-23T18:34:01.239042+0000","last_became_peered":"2024-04-23T18:34:01.239042+0000","last_unstale":"2024-04-23T18:34:55.257726+0000","last_undegraded":"2024-04-23T18:34:55.257726+0000","last_fullsized":"2024-04-23T18:34:55.257726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:45.755462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.273948+0000","last_change":"2024-04-23T18:34:55.273529+0000","last_active":"2024-04-23T18:34:55.273948+0000","last_peered":"2024-04-23T18:34:55.273948+0000","last_clean":"2024-04-23T18:34:55.273948+0000","last_became_active":"2024-04-23T18:34:55.259780+0000","last_became_peered":"2024-04-23T18:34:55.259780+0000","last_unstale":"2024-04-23T18:34:55.273948+0000","last_undegraded":"2024-04-23T18:34:55.273948+0000","last_fullsized":"2024-04-23T18:34:55.273948+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:48:56.272606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260773+0000","last_change":"2024-04-23T18:34:01.239943+0000","last_active":"2024-04-23T18:34:55.260773+0000","last_peered":"2024-04-23T18:34:55.260773+0000","last_clean":"2024-04-23T18:34:55.260773+0000","last_became_active":"2024-04-23T18:34:01.239588+0000","last_became_peered":"2024-04-23T18:34:01.239588+0000","last_unstale":"2024-04-23T18:34:55.260773+0000","last_undegraded":"2024-04-23T18:34:55.260773+0000","last_fullsized":"2024-04-23T18:34:55.260773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:51:39.344341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256091+0000","last_change":"2024-04-23T18:34:00.227372+0000","last_active":"2024-04-23T18:34:55.256091+0000","last_peered":"2024-04-23T18:34:55.256091+0000","last_clean":"2024-04-23T18:34:55.256091+0000","last_became_active":"2024-04-23T18:34:00.226709+0000","last_became_peered":"2024-04-23T18:34:00.226709+0000","last_unstale":"2024-04-23T18:34:55.256091+0000","last_undegraded":"2024-04-23T18:34:55.256091+0000","last_fullsized":"2024-04-23T18:34:55.256091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:20.047108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253087+0000","last_change":"2024-04-23T18:34:01.235389+0000","last_active":"2024-04-23T18:34:55.253087+0000","last_peered":"2024-04-23T18:34:55.253087+0000","last_clean":"2024-04-23T18:34:55.253087+0000","last_became_active":"2024-04-23T18:34:01.235026+0000","last_became_peered":"2024-04-23T18:34:01.235026+0000","last_unstale":"2024-04-23T18:34:55.253087+0000","last_undegraded":"2024-04-23T18:34:55.253087+0000","last_fullsized":"2024-04-23T18:34:55.253087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:48.849525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260852+0000","last_change":"2024-04-23T18:34:55.260852+0000","last_active":"2024-04-23T18:34:55.260852+0000","last_peered":"2024-04-23T18:34:55.260852+0000","last_clean":"2024-04-23T18:34:55.260852+0000","last_became_active":"2024-04-23T18:34:55.260452+0000","last_became_peered":"2024-04-23T18:34:55.260452+0000","last_unstale":"2024-04-23T18:34:55.260852+0000","last_undegraded":"2024-04-23T18:34:55.260852+0000","last_fullsized":"2024-04-23T18:34:55.260852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:28:13.496447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253435+0000","last_change":"2024-04-23T18:34:01.224992+0000","last_active":"2024-04-23T18:34:55.253435+0000","last_peered":"2024-04-23T18:34:55.253435+0000","last_clean":"2024-04-23T18:34:55.253435+0000","last_became_active":"2024-04-23T18:34:01.224508+0000","last_became_peered":"2024-04-23T18:34:01.224508+0000","last_unstale":"2024-04-23T18:34:55.253435+0000","last_undegraded":"2024-04-23T18:34:55.253435+0000","last_fullsized":"2024-04-23T18:34:55.253435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:24.183451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:15.634122+0000","last_change":"2024-04-23T18:34:00.234818+0000","last_active":"2024-04-23T18:35:15.634122+0000","last_peered":"2024-04-23T18:35:15.634122+0000","last_clean":"2024-04-23T18:35:15.634122+0000","last_became_active":"2024-04-23T18:34:00.234329+0000","last_became_peered":"2024-04-23T18:34:00.234329+0000","last_unstale":"2024-04-23T18:35:15.634122+0000","last_undegraded":"2024-04-23T18:35:15.634122+0000","last_fullsized":"2024-04-23T18:35:15.634122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:43:02.405694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.261073+0000","last_change":"2024-04-23T18:34:55.261073+0000","last_active":"2024-04-23T18:34:55.261073+0000","last_peered":"2024-04-23T18:34:55.261073+0000","last_clean":"2024-04-23T18:34:55.261073+0000","last_became_active":"2024-04-23T18:34:55.260655+0000","last_became_peered":"2024-04-23T18:34:55.260655+0000","last_unstale":"2024-04-23T18:34:55.261073+0000","last_undegraded":"2024-04-23T18:34:55.261073+0000","last_fullsized":"2024-04-23T18:34:55.261073+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:37:36.797785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260771+0000","last_change":"2024-04-23T18:34:01.234184+0000","last_active":"2024-04-23T18:34:55.260771+0000","last_peered":"2024-04-23T18:34:55.260771+0000","last_clean":"2024-04-23T18:34:55.260771+0000","last_became_active":"2024-04-23T18:34:01.233846+0000","last_became_peered":"2024-04-23T18:34:01.233846+0000","last_unstale":"2024-04-23T18:34:55.260771+0000","last_undegraded":"2024-04-23T18:34:55.260771+0000","last_fullsized":"2024-04-23T18:34:55.260771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:41.771495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:13.637122+0000","last_change":"2024-04-23T18:34:00.227862+0000","last_active":"2024-04-23T18:35:13.637122+0000","last_peered":"2024-04-23T18:35:13.637122+0000","last_clean":"2024-04-23T18:35:13.637122+0000","last_became_active":"2024-04-23T18:34:00.227515+0000","last_became_peered":"2024-04-23T18:34:00.227515+0000","last_unstale":"2024-04-23T18:35:13.637122+0000","last_undegraded":"2024-04-23T18:35:13.637122+0000","last_fullsized":"2024-04-23T18:35:13.637122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:18.595690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286382+0000","last_change":"2024-04-23T18:34:01.236338+0000","last_active":"2024-04-23T18:34:55.286382+0000","last_peered":"2024-04-23T18:34:55.286382+0000","last_clean":"2024-04-23T18:34:55.286382+0000","last_became_active":"2024-04-23T18:34:01.235795+0000","last_became_peered":"2024-04-23T18:34:01.235795+0000","last_unstale":"2024-04-23T18:34:55.286382+0000","last_undegraded":"2024-04-23T18:34:55.286382+0000","last_fullsized":"2024-04-23T18:34:55.286382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:45:01.928793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.287511+0000","last_change":"2024-04-23T18:34:00.221851+0000","last_active":"2024-04-23T18:34:55.287511+0000","last_peered":"2024-04-23T18:34:55.287511+0000","last_clean":"2024-04-23T18:34:55.287511+0000","last_became_active":"2024-04-23T18:34:00.221513+0000","last_became_peered":"2024-04-23T18:34:00.221513+0000","last_unstale":"2024-04-23T18:34:55.287511+0000","last_undegraded":"2024-04-23T18:34:55.287511+0000","last_fullsized":"2024-04-23T18:34:55.287511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:15:04.608759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.257868+0000","last_change":"2024-04-23T18:32:48.218931+0000","last_active":"2024-04-23T18:34:55.257868+0000","last_peered":"2024-04-23T18:34:55.257868+0000","last_clean":"2024-04-23T18:34:55.257868+0000","last_became_active":"2024-04-23T18:32:48.218484+0000","last_became_peered":"2024-04-23T18:32:48.218484+0000","last_unstale":"2024-04-23T18:34:55.257868+0000","last_undegraded":"2024-04-23T18:34:55.257868+0000","last_fullsized":"2024-04-23T18:34:55.257868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_clean_scrub_stamp":"2024-04-23T18:29:21.029319+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:03:42.234165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255404+0000","last_change":"2024-04-23T18:34:01.220346+0000","last_active":"2024-04-23T18:34:55.255404+0000","last_peered":"2024-04-23T18:34:55.255404+0000","last_clean":"2024-04-23T18:34:55.255404+0000","last_became_active":"2024-04-23T18:34:01.219991+0000","last_became_peered":"2024-04-23T18:34:01.219991+0000","last_unstale":"2024-04-23T18:34:55.255404+0000","last_undegraded":"2024-04-23T18:34:55.255404+0000","last_fullsized":"2024-04-23T18:34:55.255404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:29:00.662700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.257995+0000","last_change":"2024-04-23T18:34:00.229625+0000","last_active":"2024-04-23T18:34:55.257995+0000","last_peered":"2024-04-23T18:34:55.257995+0000","last_clean":"2024-04-23T18:34:55.257995+0000","last_became_active":"2024-04-23T18:34:00.229204+0000","last_became_peered":"2024-04-23T18:34:00.229204+0000","last_unstale":"2024-04-23T18:34:55.257995+0000","last_undegraded":"2024-04-23T18:34:55.257995+0000","last_fullsized":"2024-04-23T18:34:55.257995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:42.370870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256319+0000","last_change":"2024-04-23T18:34:00.236000+0000","last_active":"2024-04-23T18:34:55.256319+0000","last_peered":"2024-04-23T18:34:55.256319+0000","last_clean":"2024-04-23T18:34:55.256319+0000","last_became_active":"2024-04-23T18:34:00.235561+0000","last_became_peered":"2024-04-23T18:34:00.235561+0000","last_unstale":"2024-04-23T18:34:55.256319+0000","last_undegraded":"2024-04-23T18:34:55.256319+0000","last_fullsized":"2024-04-23T18:34:55.256319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:07.565489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256199+0000","last_change":"2024-04-23T18:34:01.227955+0000","last_active":"2024-04-23T18:34:55.256199+0000","last_peered":"2024-04-23T18:34:55.256199+0000","last_clean":"2024-04-23T18:34:55.256199+0000","last_became_active":"2024-04-23T18:34:01.225992+0000","last_became_peered":"2024-04-23T18:34:01.225992+0000","last_unstale":"2024-04-23T18:34:55.256199+0000","last_undegraded":"2024-04-23T18:34:55.256199+0000","last_fullsized":"2024-04-23T18:34:55.256199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:05:55.147008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255849+0000","last_change":"2024-04-23T18:34:00.230133+0000","last_active":"2024-04-23T18:34:55.255849+0000","last_peered":"2024-04-23T18:34:55.255849+0000","last_clean":"2024-04-23T18:34:55.255849+0000","last_became_active":"2024-04-23T18:34:00.229596+0000","last_became_peered":"2024-04-23T18:34:00.229596+0000","last_unstale":"2024-04-23T18:34:55.255849+0000","last_undegraded":"2024-04-23T18:34:55.255849+0000","last_fullsized":"2024-04-23T18:34:55.255849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:23:06.064313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286678+0000","last_change":"2024-04-23T18:34:01.229066+0000","last_active":"2024-04-23T18:34:55.286678+0000","last_peered":"2024-04-23T18:34:55.286678+0000","last_clean":"2024-04-23T18:34:55.286678+0000","last_became_active":"2024-04-23T18:34:01.228711+0000","last_became_peered":"2024-04-23T18:34:01.228711+0000","last_unstale":"2024-04-23T18:34:55.286678+0000","last_undegraded":"2024-04-23T18:34:55.286678+0000","last_fullsized":"2024-04-23T18:34:55.286678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:58:30.601270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.287316+0000","last_change":"2024-04-23T18:34:00.231227+0000","last_active":"2024-04-23T18:34:55.287316+0000","last_peered":"2024-04-23T18:34:55.287316+0000","last_clean":"2024-04-23T18:34:55.287316+0000","last_became_active":"2024-04-23T18:34:00.230843+0000","last_became_peered":"2024-04-23T18:34:00.230843+0000","last_unstale":"2024-04-23T18:34:55.287316+0000","last_undegraded":"2024-04-23T18:34:55.287316+0000","last_fullsized":"2024-04-23T18:34:55.287316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:55.416591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256753+0000","last_change":"2024-04-23T18:34:01.233164+0000","last_active":"2024-04-23T18:34:55.256753+0000","last_peered":"2024-04-23T18:34:55.256753+0000","last_clean":"2024-04-23T18:34:55.256753+0000","last_became_active":"2024-04-23T18:34:01.232827+0000","last_became_peered":"2024-04-23T18:34:01.232827+0000","last_unstale":"2024-04-23T18:34:55.256753+0000","last_undegraded":"2024-04-23T18:34:55.256753+0000","last_fullsized":"2024-04-23T18:34:55.256753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:09.635515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286425+0000","last_change":"2024-04-23T18:34:00.225651+0000","last_active":"2024-04-23T18:34:55.286425+0000","last_peered":"2024-04-23T18:34:55.286425+0000","last_clean":"2024-04-23T18:34:55.286425+0000","last_became_active":"2024-04-23T18:34:00.225001+0000","last_became_peered":"2024-04-23T18:34:00.225001+0000","last_unstale":"2024-04-23T18:34:55.286425+0000","last_undegraded":"2024-04-23T18:34:55.286425+0000","last_fullsized":"2024-04-23T18:34:55.286425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:27:44.265388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255780+0000","last_change":"2024-04-23T18:34:01.224131+0000","last_active":"2024-04-23T18:34:55.255780+0000","last_peered":"2024-04-23T18:34:55.255780+0000","last_clean":"2024-04-23T18:34:55.255780+0000","last_became_active":"2024-04-23T18:34:01.223742+0000","last_became_peered":"2024-04-23T18:34:01.223742+0000","last_unstale":"2024-04-23T18:34:55.255780+0000","last_undegraded":"2024-04-23T18:34:55.255780+0000","last_fullsized":"2024-04-23T18:34:55.255780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:29:26.430964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286062+0000","last_change":"2024-04-23T18:34:00.235789+0000","last_active":"2024-04-23T18:34:55.286062+0000","last_peered":"2024-04-23T18:34:55.286062+0000","last_clean":"2024-04-23T18:34:55.286062+0000","last_became_active":"2024-04-23T18:34:00.235452+0000","last_became_peered":"2024-04-23T18:34:00.235452+0000","last_unstale":"2024-04-23T18:34:55.286062+0000","last_undegraded":"2024-04-23T18:34:55.286062+0000","last_fullsized":"2024-04-23T18:34:55.286062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:21.090121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.252908+0000","last_change":"2024-04-23T18:34:01.226156+0000","last_active":"2024-04-23T18:34:55.252908+0000","last_peered":"2024-04-23T18:34:55.252908+0000","last_clean":"2024-04-23T18:34:55.252908+0000","last_became_active":"2024-04-23T18:34:01.225817+0000","last_became_peered":"2024-04-23T18:34:01.225817+0000","last_unstale":"2024-04-23T18:34:55.252908+0000","last_undegraded":"2024-04-23T18:34:55.252908+0000","last_fullsized":"2024-04-23T18:34:55.252908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:36.337139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255363+0000","last_change":"2024-04-23T18:34:00.235116+0000","last_active":"2024-04-23T18:34:55.255363+0000","last_peered":"2024-04-23T18:34:55.255363+0000","last_clean":"2024-04-23T18:34:55.255363+0000","last_became_active":"2024-04-23T18:34:00.234521+0000","last_became_peered":"2024-04-23T18:34:00.234521+0000","last_unstale":"2024-04-23T18:34:55.255363+0000","last_undegraded":"2024-04-23T18:34:55.255363+0000","last_fullsized":"2024-04-23T18:34:55.255363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:51.061164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286352+0000","last_change":"2024-04-23T18:34:01.226445+0000","last_active":"2024-04-23T18:34:55.286352+0000","last_peered":"2024-04-23T18:34:55.286352+0000","last_clean":"2024-04-23T18:34:55.286352+0000","last_became_active":"2024-04-23T18:34:01.225875+0000","last_became_peered":"2024-04-23T18:34:01.225875+0000","last_unstale":"2024-04-23T18:34:55.286352+0000","last_undegraded":"2024-04-23T18:34:55.286352+0000","last_fullsized":"2024-04-23T18:34:55.286352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:48:38.712502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255934+0000","last_change":"2024-04-23T18:34:00.231918+0000","last_active":"2024-04-23T18:34:55.255934+0000","last_peered":"2024-04-23T18:34:55.255934+0000","last_clean":"2024-04-23T18:34:55.255934+0000","last_became_active":"2024-04-23T18:34:00.231550+0000","last_became_peered":"2024-04-23T18:34:00.231550+0000","last_unstale":"2024-04-23T18:34:55.255934+0000","last_undegraded":"2024-04-23T18:34:55.255934+0000","last_fullsized":"2024-04-23T18:34:55.255934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:52.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258228+0000","last_change":"2024-04-23T18:34:01.236360+0000","last_active":"2024-04-23T18:34:55.258228+0000","last_peered":"2024-04-23T18:34:55.258228+0000","last_clean":"2024-04-23T18:34:55.258228+0000","last_became_active":"2024-04-23T18:34:01.236056+0000","last_became_peered":"2024-04-23T18:34:01.236056+0000","last_unstale":"2024-04-23T18:34:55.258228+0000","last_undegraded":"2024-04-23T18:34:55.258228+0000","last_fullsized":"2024-04-23T18:34:55.258228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:54.535158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:35:13.646507+0000","last_change":"2024-04-23T18:34:00.233159+0000","last_active":"2024-04-23T18:35:13.646507+0000","last_peered":"2024-04-23T18:35:13.646507+0000","last_clean":"2024-04-23T18:35:13.646507+0000","last_became_active":"2024-04-23T18:34:00.232728+0000","last_became_peered":"2024-04-23T18:34:00.232728+0000","last_unstale":"2024-04-23T18:35:13.646507+0000","last_undegraded":"2024-04-23T18:35:13.646507+0000","last_fullsized":"2024-04-23T18:35:13.646507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:23.498057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258882+0000","last_change":"2024-04-23T18:34:01.226134+0000","last_active":"2024-04-23T18:34:55.258882+0000","last_peered":"2024-04-23T18:34:55.258882+0000","last_clean":"2024-04-23T18:34:55.258882+0000","last_became_active":"2024-04-23T18:34:01.225759+0000","last_became_peered":"2024-04-23T18:34:01.225759+0000","last_unstale":"2024-04-23T18:34:55.258882+0000","last_undegraded":"2024-04-23T18:34:55.258882+0000","last_fullsized":"2024-04-23T18:34:55.258882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:24.770595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260552+0000","last_change":"2024-04-23T18:34:00.230054+0000","last_active":"2024-04-23T18:34:55.260552+0000","last_peered":"2024-04-23T18:34:55.260552+0000","last_clean":"2024-04-23T18:34:55.260552+0000","last_became_active":"2024-04-23T18:34:00.229621+0000","last_became_peered":"2024-04-23T18:34:00.229621+0000","last_unstale":"2024-04-23T18:34:55.260552+0000","last_undegraded":"2024-04-23T18:34:55.260552+0000","last_fullsized":"2024-04-23T18:34:55.260552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:13:59.317282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.259118+0000","last_change":"2024-04-23T18:34:01.240053+0000","last_active":"2024-04-23T18:34:55.259118+0000","last_peered":"2024-04-23T18:34:55.259118+0000","last_clean":"2024-04-23T18:34:55.259118+0000","last_became_active":"2024-04-23T18:34:01.239716+0000","last_became_peered":"2024-04-23T18:34:01.239716+0000","last_unstale":"2024-04-23T18:34:55.259118+0000","last_undegraded":"2024-04-23T18:34:55.259118+0000","last_fullsized":"2024-04-23T18:34:55.259118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:54.874374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256530+0000","last_change":"2024-04-23T18:34:00.227914+0000","last_active":"2024-04-23T18:34:55.256530+0000","last_peered":"2024-04-23T18:34:55.256530+0000","last_clean":"2024-04-23T18:34:55.256530+0000","last_became_active":"2024-04-23T18:34:00.227540+0000","last_became_peered":"2024-04-23T18:34:00.227540+0000","last_unstale":"2024-04-23T18:34:55.256530+0000","last_undegraded":"2024-04-23T18:34:55.256530+0000","last_fullsized":"2024-04-23T18:34:55.256530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:18.442397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258982+0000","last_change":"2024-04-23T18:34:01.227319+0000","last_active":"2024-04-23T18:34:55.258982+0000","last_peered":"2024-04-23T18:34:55.258982+0000","last_clean":"2024-04-23T18:34:55.258982+0000","last_became_active":"2024-04-23T18:34:01.226969+0000","last_became_peered":"2024-04-23T18:34:01.226969+0000","last_unstale":"2024-04-23T18:34:55.258982+0000","last_undegraded":"2024-04-23T18:34:55.258982+0000","last_fullsized":"2024-04-23T18:34:55.258982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:19.302388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253082+0000","last_change":"2024-04-23T18:34:01.234466+0000","last_active":"2024-04-23T18:34:55.253082+0000","last_peered":"2024-04-23T18:34:55.253082+0000","last_clean":"2024-04-23T18:34:55.253082+0000","last_became_active":"2024-04-23T18:34:01.234127+0000","last_became_peered":"2024-04-23T18:34:01.234127+0000","last_unstale":"2024-04-23T18:34:55.253082+0000","last_undegraded":"2024-04-23T18:34:55.253082+0000","last_fullsized":"2024-04-23T18:34:55.253082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:51:51.912320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255255+0000","last_change":"2024-04-23T18:34:01.229707+0000","last_active":"2024-04-23T18:34:55.255255+0000","last_peered":"2024-04-23T18:34:55.255255+0000","last_clean":"2024-04-23T18:34:55.255255+0000","last_became_active":"2024-04-23T18:34:01.229311+0000","last_became_peered":"2024-04-23T18:34:01.229311+0000","last_unstale":"2024-04-23T18:34:55.255255+0000","last_undegraded":"2024-04-23T18:34:55.255255+0000","last_fullsized":"2024-04-23T18:34:55.255255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:15:51.972436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255233+0000","last_change":"2024-04-23T18:34:01.227285+0000","last_active":"2024-04-23T18:34:55.255233+0000","last_peered":"2024-04-23T18:34:55.255233+0000","last_clean":"2024-04-23T18:34:55.255233+0000","last_became_active":"2024-04-23T18:34:01.226960+0000","last_became_peered":"2024-04-23T18:34:01.226960+0000","last_unstale":"2024-04-23T18:34:55.255233+0000","last_undegraded":"2024-04-23T18:34:55.255233+0000","last_fullsized":"2024-04-23T18:34:55.255233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:50.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.258290+0000","last_change":"2024-04-23T18:34:01.230697+0000","last_active":"2024-04-23T18:34:55.258290+0000","last_peered":"2024-04-23T18:34:55.258290+0000","last_clean":"2024-04-23T18:34:55.258290+0000","last_became_active":"2024-04-23T18:34:01.230352+0000","last_became_peered":"2024-04-23T18:34:01.230352+0000","last_unstale":"2024-04-23T18:34:55.258290+0000","last_undegraded":"2024-04-23T18:34:55.258290+0000","last_fullsized":"2024-04-23T18:34:55.258290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:35:10.609773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.256294+0000","last_change":"2024-04-23T18:34:01.226970+0000","last_active":"2024-04-23T18:34:55.256294+0000","last_peered":"2024-04-23T18:34:55.256294+0000","last_clean":"2024-04-23T18:34:55.256294+0000","last_became_active":"2024-04-23T18:34:01.224151+0000","last_became_peered":"2024-04-23T18:34:01.224151+0000","last_unstale":"2024-04-23T18:34:55.256294+0000","last_undegraded":"2024-04-23T18:34:55.256294+0000","last_fullsized":"2024-04-23T18:34:55.256294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:36:15.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.253136+0000","last_change":"2024-04-23T18:34:01.234194+0000","last_active":"2024-04-23T18:34:55.253136+0000","last_peered":"2024-04-23T18:34:55.253136+0000","last_clean":"2024-04-23T18:34:55.253136+0000","last_became_active":"2024-04-23T18:34:01.232404+0000","last_became_peered":"2024-04-23T18:34:01.232404+0000","last_unstale":"2024-04-23T18:34:55.253136+0000","last_undegraded":"2024-04-23T18:34:55.253136+0000","last_fullsized":"2024-04-23T18:34:55.253136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:30.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255670+0000","last_change":"2024-04-23T18:34:01.219507+0000","last_active":"2024-04-23T18:34:55.255670+0000","last_peered":"2024-04-23T18:34:55.255670+0000","last_clean":"2024-04-23T18:34:55.255670+0000","last_became_active":"2024-04-23T18:34:01.218997+0000","last_became_peered":"2024-04-23T18:34:01.218997+0000","last_unstale":"2024-04-23T18:34:55.255670+0000","last_undegraded":"2024-04-23T18:34:55.255670+0000","last_fullsized":"2024-04-23T18:34:55.255670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:38.233652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286591+0000","last_change":"2024-04-23T18:34:01.231846+0000","last_active":"2024-04-23T18:34:55.286591+0000","last_peered":"2024-04-23T18:34:55.286591+0000","last_clean":"2024-04-23T18:34:55.286591+0000","last_became_active":"2024-04-23T18:34:01.231467+0000","last_became_peered":"2024-04-23T18:34:01.231467+0000","last_unstale":"2024-04-23T18:34:55.286591+0000","last_undegraded":"2024-04-23T18:34:55.286591+0000","last_fullsized":"2024-04-23T18:34:55.286591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:37:27.891180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.260623+0000","last_change":"2024-04-23T18:34:01.237086+0000","last_active":"2024-04-23T18:34:55.260623+0000","last_peered":"2024-04-23T18:34:55.260623+0000","last_clean":"2024-04-23T18:34:55.260623+0000","last_became_active":"2024-04-23T18:34:01.236610+0000","last_became_peered":"2024-04-23T18:34:01.236610+0000","last_unstale":"2024-04-23T18:34:55.260623+0000","last_undegraded":"2024-04-23T18:34:55.260623+0000","last_fullsized":"2024-04-23T18:34:55.260623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:24.916553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286027+0000","last_change":"2024-04-23T18:34:01.231091+0000","last_active":"2024-04-23T18:34:55.286027+0000","last_peered":"2024-04-23T18:34:55.286027+0000","last_clean":"2024-04-23T18:34:55.286027+0000","last_became_active":"2024-04-23T18:34:01.230731+0000","last_became_peered":"2024-04-23T18:34:01.230731+0000","last_unstale":"2024-04-23T18:34:55.286027+0000","last_undegraded":"2024-04-23T18:34:55.286027+0000","last_fullsized":"2024-04-23T18:34:55.286027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:22.771515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.286699+0000","last_change":"2024-04-23T18:34:01.231820+0000","last_active":"2024-04-23T18:34:55.286699+0000","last_peered":"2024-04-23T18:34:55.286699+0000","last_clean":"2024-04-23T18:34:55.286699+0000","last_became_active":"2024-04-23T18:34:01.230320+0000","last_became_peered":"2024-04-23T18:34:01.230320+0000","last_unstale":"2024-04-23T18:34:55.286699+0000","last_undegraded":"2024-04-23T18:34:55.286699+0000","last_fullsized":"2024-04-23T18:34:55.286699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:27:35.324323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.255907+0000","last_change":"2024-04-23T18:34:01.228723+0000","last_active":"2024-04-23T18:34:55.255907+0000","last_peered":"2024-04-23T18:34:55.255907+0000","last_clean":"2024-04-23T18:34:55.255907+0000","last_became_active":"2024-04-23T18:34:01.227500+0000","last_became_peered":"2024-04-23T18:34:01.227500+0000","last_unstale":"2024-04-23T18:34:55.255907+0000","last_undegraded":"2024-04-23T18:34:55.255907+0000","last_fullsized":"2024-04-23T18:34:55.255907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:17.908021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:34:55.252992+0000","last_change":"2024-04-23T18:34:01.227541+0000","last_active":"2024-04-23T18:34:55.252992+0000","last_peered":"2024-04-23T18:34:55.252992+0000","last_clean":"2024-04-23T18:34:55.252992+0000","last_became_active":"2024-04-23T18:34:01.227201+0000","last_became_peered":"2024-04-23T18:34:01.227201+0000","last_unstale":"2024-04-23T18:34:55.252992+0000","last_undegraded":"2024-04-23T18:34:55.252992+0000","last_fullsized":"2024-04-23T18:34:55.252992+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:25:24.886850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282171,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66669,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":740,"num_read_kb":442,"num_write":236,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":200007,"data_compressed":5622,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":462787,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":1112778,"data_compressed":43551,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":462214,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":462480,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1055110,"data_compressed":41677,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":461931,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":522309,"data_compressed":7665,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":463661,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":523311,"data_compressed":7665,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":461756,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":461792,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":1054788,"data_compressed":41677,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34247,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34247,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34247,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61217,"data_compressed":1874,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62219,"data_compressed":1874,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61201,"data_compressed":1874,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:35:22.618 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-23T18:35:22.618 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-23T18:35:22.618 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-23T18:35:22.619 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-23T18:35:22.861 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:23.258 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:23 smithi132 ceph-mon[29178]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:35:23.258 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:23 smithi132 ceph-mon[29178]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:35:23.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:23 smithi148 ceph-mon[32311]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:35:23.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:23 smithi148 ceph-mon[32311]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:35:23.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:23 smithi157 ceph-mon[32580]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:35:23.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:23 smithi157 ceph-mon[32580]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:35:24.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.577+0000 7f836034c700 1 -- 172.21.15.132:0/314969917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8358102300 msgr2=0x7f835810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:24.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.577+0000 7f836034c700 1 --2- 172.21.15.132:0/314969917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8358102300 0x7f835810e7f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8354009990 tx=0x7f8354009ca0 comp rx=0 tx=0).stop 2024-04-23T18:35:24.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.577+0000 7f836034c700 1 -- 172.21.15.132:0/314969917 shutdown_connections 2024-04-23T18:35:24.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.577+0000 7f836034c700 1 --2- 172.21.15.132:0/314969917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8358102300 0x7f835810e7f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:24.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.577+0000 7f836034c700 1 --2- 172.21.15.132:0/314969917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8358107990 0x7f8358107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:24.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.577+0000 7f836034c700 1 --2- 172.21.15.132:0/314969917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8358101940 0x7f8358101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:24.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.577+0000 7f836034c700 1 -- 172.21.15.132:0/314969917 >> 172.21.15.132:0/314969917 conn(0x7f83580788f0 msgr2=0x7f8358078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:24.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.577+0000 7f836034c700 1 -- 172.21.15.132:0/314969917 shutdown_connections 2024-04-23T18:35:24.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.577+0000 7f836034c700 1 -- 172.21.15.132:0/314969917 wait complete. 2024-04-23T18:35:24.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f836034c700 1 Processor -- start 2024-04-23T18:35:24.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f836034c700 1 -- start start 2024-04-23T18:35:24.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f836034c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8358101940 0x7f8358074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:24.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f836034c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8358102300 0x7f835806c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f836034c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8358107990 0x7f835806ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f836034c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8358111af0 con 0x7f8358101940 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f836034c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8358111970 con 0x7f8358107990 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f836034c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8358111c70 con 0x7f8358102300 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f835e0e8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8358101940 0x7f8358074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f835e0e8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8358101940 0x7f8358074580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49076/0 (socket says 172.21.15.132:49076) 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f835e0e8700 1 -- 172.21.15.132:0/1752518412 learned_addr learned my addr 172.21.15.132:0/1752518412 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f835d8e7700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8358102300 0x7f835806c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.578+0000 7f835e8e9700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8358107990 0x7f835806ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.579+0000 7f835e0e8700 1 -- 172.21.15.132:0/1752518412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8358102300 msgr2=0x7f835806c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.579+0000 7f835e0e8700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8358102300 0x7f835806c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.579+0000 7f835e0e8700 1 -- 172.21.15.132:0/1752518412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8358107990 msgr2=0x7f835806ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.579+0000 7f835e0e8700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8358107990 0x7f835806ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:24.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.579+0000 7f835e0e8700 1 -- 172.21.15.132:0/1752518412 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835806d390 con 0x7f8358101940 2024-04-23T18:35:24.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.579+0000 7f835e8e9700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8358107990 0x7f835806ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:24.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.579+0000 7f835e0e8700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8358101940 0x7f8358074580 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8350007e20 tx=0x7f835000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:24.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.579+0000 7f834f7fe700 1 -- 172.21.15.132:0/1752518412 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350016020 con 0x7f8358101940 2024-04-23T18:35:24.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.579+0000 7f836034c700 1 -- 172.21.15.132:0/1752518412 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83581afff0 con 0x7f8358101940 2024-04-23T18:35:24.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.579+0000 7f834f7fe700 1 -- 172.21.15.132:0/1752518412 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f835000ad40 con 0x7f8358101940 2024-04-23T18:35:24.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.579+0000 7f834f7fe700 1 -- 172.21.15.132:0/1752518412 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350011670 con 0x7f8358101940 2024-04-23T18:35:24.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.579+0000 7f836034c700 1 -- 172.21.15.132:0/1752518412 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83581b04a0 con 0x7f8358101940 2024-04-23T18:35:24.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.580+0000 7f836034c700 1 -- 172.21.15.132:0/1752518412 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8358103060 con 0x7f8358101940 2024-04-23T18:35:24.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.581+0000 7f834f7fe700 1 -- 172.21.15.132:0/1752518412 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f8350019020 con 0x7f8358101940 2024-04-23T18:35:24.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.582+0000 7f834f7fe700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f83440a3c40 0x7f83440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:24.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.582+0000 7f835d8e7700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f83440a3c40 0x7f83440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:24.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.582+0000 7f834f7fe700 1 -- 172.21.15.132:0/1752518412 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f83500d05a0 con 0x7f8358101940 2024-04-23T18:35:24.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.582+0000 7f835d8e7700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f83440a3c40 0x7f83440a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f835806dd10 tx=0x7f834800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:24.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.586+0000 7f834f7fe700 1 -- 172.21.15.132:0/1752518412 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8350020070 con 0x7f8358101940 2024-04-23T18:35:24.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.831+0000 7f836034c700 1 -- 172.21.15.132:0/1752518412 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8358069730 con 0x7f8358101940 2024-04-23T18:35:24.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.832+0000 7f834f7fe700 1 -- 172.21.15.132:0/1752518412 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8358069730 con 0x7f8358101940 2024-04-23T18:35:24.824 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:24.824 INFO:teuthology.orchestra.run.smithi132.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T18:35:24.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.834+0000 7f836034c700 1 -- 172.21.15.132:0/1752518412 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f83440a3c40 msgr2=0x7f83440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:24.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.834+0000 7f836034c700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f83440a3c40 0x7f83440a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f835806dd10 tx=0x7f834800b3c0 comp rx=0 tx=0).stop 2024-04-23T18:35:24.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.834+0000 7f836034c700 1 -- 172.21.15.132:0/1752518412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8358101940 msgr2=0x7f8358074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:24.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.834+0000 7f836034c700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8358101940 0x7f8358074580 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8350007e20 tx=0x7f835000a340 comp rx=0 tx=0).stop 2024-04-23T18:35:24.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.834+0000 7f836034c700 1 -- 172.21.15.132:0/1752518412 shutdown_connections 2024-04-23T18:35:24.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.834+0000 7f836034c700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8358107990 0x7f835806ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:24.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.834+0000 7f836034c700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f83440a3c40 0x7f83440a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:24.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.834+0000 7f836034c700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8358101940 0x7f8358074580 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:24.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.834+0000 7f836034c700 1 --2- 172.21.15.132:0/1752518412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8358102300 0x7f835806c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:24.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.834+0000 7f836034c700 1 -- 172.21.15.132:0/1752518412 >> 172.21.15.132:0/1752518412 conn(0x7f83580788f0 msgr2=0x7f835806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:24.827 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.834+0000 7f836034c700 1 -- 172.21.15.132:0/1752518412 shutdown_connections 2024-04-23T18:35:24.827 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:24.834+0000 7f836034c700 1 -- 172.21.15.132:0/1752518412 wait complete. 2024-04-23T18:35:25.474 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:25 smithi132 ceph-mon[29178]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:25.474 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:25 smithi132 ceph-mon[29178]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T18:35:25.474 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:25 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1752518412' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T18:35:25.504 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-23T18:35:25.505 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:35:25.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:25 smithi148 ceph-mon[32311]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:25.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:25 smithi148 ceph-mon[32311]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T18:35:25.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:25 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1752518412' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T18:35:25.542 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:25 smithi157 ceph-mon[32580]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:25.542 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:25 smithi157 ceph-mon[32580]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T18:35:25.542 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:25 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1752518412' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T18:35:25.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.616+0000 7f881d1a2640 1 -- 172.21.15.132:0/4242311882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8818101fc0 msgr2=0x7f88181023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:25.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.616+0000 7f881d1a2640 1 --2- 172.21.15.132:0/4242311882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8818101fc0 0x7f88181023e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8808009900 tx=0x7f880802a410 comp rx=0 tx=0).stop 2024-04-23T18:35:25.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.616+0000 7f881d1a2640 1 -- 172.21.15.132:0/4242311882 shutdown_connections 2024-04-23T18:35:25.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.616+0000 7f881d1a2640 1 --2- 172.21.15.132:0/4242311882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8818103b70 0x7f881810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.616+0000 7f881d1a2640 1 --2- 172.21.15.132:0/4242311882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88181031b0 0x7f8818103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.616+0000 7f881d1a2640 1 --2- 172.21.15.132:0/4242311882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8818101fc0 0x7f88181023e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.616+0000 7f881d1a2640 1 -- 172.21.15.132:0/4242311882 >> 172.21.15.132:0/4242311882 conn(0x7f88180fd530 msgr2=0x7f88180ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.616+0000 7f881d1a2640 1 -- 172.21.15.132:0/4242311882 shutdown_connections 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.616+0000 7f881d1a2640 1 -- 172.21.15.132:0/4242311882 wait complete. 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f881d1a2640 1 Processor -- start 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f881d1a2640 1 -- start start 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f881d1a2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8818101fc0 0x7f88181981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f881d1a2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88181031b0 0x7f8818198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f881d1a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8818103b70 0x7f881819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f881d1a2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f881810dac0 con 0x7f8818101fc0 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f881d1a2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f881810d940 con 0x7f88181031b0 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f881d1a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f881810dc40 con 0x7f8818103b70 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f8816d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8818101fc0 0x7f88181981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f8816d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8818101fc0 0x7f88181981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49104/0 (socket says 172.21.15.132:49104) 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f8816d9d640 1 -- 172.21.15.132:0/2445488321 learned_addr learned my addr 172.21.15.132:0/2445488321 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f8816d9d640 1 -- 172.21.15.132:0/2445488321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8818103b70 msgr2=0x7f881819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:25.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f8816d9d640 1 --2- 172.21.15.132:0/2445488321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8818103b70 0x7f881819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f8816d9d640 1 -- 172.21.15.132:0/2445488321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88181031b0 msgr2=0x7f8818198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:25.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f8816d9d640 1 --2- 172.21.15.132:0/2445488321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88181031b0 0x7f8818198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f8816d9d640 1 -- 172.21.15.132:0/2445488321 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881819fe10 con 0x7f8818101fc0 2024-04-23T18:35:25.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f8816d9d640 1 --2- 172.21.15.132:0/2445488321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8818101fc0 0x7f88181981f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8808006010 tx=0x7f88080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:25.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f8814d99640 1 -- 172.21.15.132:0/2445488321 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880803f030 con 0x7f8818101fc0 2024-04-23T18:35:25.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f8814d99640 1 -- 172.21.15.132:0/2445488321 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8808003cb0 con 0x7f8818101fc0 2024-04-23T18:35:25.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f881d1a2640 1 -- 172.21.15.132:0/2445488321 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88181a00a0 con 0x7f8818101fc0 2024-04-23T18:35:25.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f8814d99640 1 -- 172.21.15.132:0/2445488321 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808035070 con 0x7f8818101fc0 2024-04-23T18:35:25.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.617+0000 7f881d1a2640 1 -- 172.21.15.132:0/2445488321 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88181a0590 con 0x7f8818101fc0 2024-04-23T18:35:25.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.618+0000 7f881d1a2640 1 -- 172.21.15.132:0/2445488321 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8818109300 con 0x7f8818101fc0 2024-04-23T18:35:25.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.619+0000 7f8814d99640 1 -- 172.21.15.132:0/2445488321 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f8808042030 con 0x7f8818101fc0 2024-04-23T18:35:25.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.620+0000 7f8814d99640 1 --2- 172.21.15.132:0/2445488321 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f88040a3bb0 0x7f88040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:25.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.620+0000 7f880ffff640 1 --2- 172.21.15.132:0/2445488321 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f88040a3bb0 0x7f88040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:25.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.620+0000 7f8814d99640 1 -- 172.21.15.132:0/2445488321 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f88080f0b00 con 0x7f8818101fc0 2024-04-23T18:35:25.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.620+0000 7f880ffff640 1 --2- 172.21.15.132:0/2445488321 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f88040a3bb0 0x7f88040a6070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8800005fd0 tx=0x7f8800005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:25.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.625+0000 7f8814d99640 1 -- 172.21.15.132:0/2445488321 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88080bd4d0 con 0x7f8818101fc0 2024-04-23T18:35:25.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.803+0000 7f881d1a2640 1 -- 172.21.15.132:0/2445488321 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8818062bf0 con 0x7f8818101fc0 2024-04-23T18:35:25.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.804+0000 7f8814d99640 1 -- 172.21.15.132:0/2445488321 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22800 (secure 0 0 0) 0x7f88080bcdd0 con 0x7f8818101fc0 2024-04-23T18:35:25.797 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:25.797 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":88,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:34:55.248852+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:35:25.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.807+0000 7f881d1a2640 1 -- 172.21.15.132:0/2445488321 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f88040a3bb0 msgr2=0x7f88040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:25.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.807+0000 7f881d1a2640 1 --2- 172.21.15.132:0/2445488321 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f88040a3bb0 0x7f88040a6070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8800005fd0 tx=0x7f8800005850 comp rx=0 tx=0).stop 2024-04-23T18:35:25.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.807+0000 7f881d1a2640 1 -- 172.21.15.132:0/2445488321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8818101fc0 msgr2=0x7f88181981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:25.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.807+0000 7f881d1a2640 1 --2- 172.21.15.132:0/2445488321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8818101fc0 0x7f88181981f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8808006010 tx=0x7f88080042f0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.807+0000 7f881d1a2640 1 -- 172.21.15.132:0/2445488321 shutdown_connections 2024-04-23T18:35:25.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.807+0000 7f881d1a2640 1 --2- 172.21.15.132:0/2445488321 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f88040a3bb0 0x7f88040a6070 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.807+0000 7f881d1a2640 1 --2- 172.21.15.132:0/2445488321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8818103b70 0x7f881819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.807+0000 7f881d1a2640 1 --2- 172.21.15.132:0/2445488321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88181031b0 0x7f8818198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.807+0000 7f881d1a2640 1 --2- 172.21.15.132:0/2445488321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8818101fc0 0x7f88181981f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.807+0000 7f881d1a2640 1 -- 172.21.15.132:0/2445488321 >> 172.21.15.132:0/2445488321 conn(0x7f88180fd530 msgr2=0x7f88181064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:25.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.807+0000 7f881d1a2640 1 -- 172.21.15.132:0/2445488321 shutdown_connections 2024-04-23T18:35:25.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.807+0000 7f881d1a2640 1 -- 172.21.15.132:0/2445488321 wait complete. 2024-04-23T18:35:25.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:29:20.132451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:33:58.407521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:33:59.518206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:35:25.811 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:35:25.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.920+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2487245886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf4102040 msgr2=0x7fdbf41024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:25.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.920+0000 7fdbfb57d640 1 --2- 172.21.15.132:0/2487245886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf4102040 0x7fdbf41024c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009900 tx=0x7fdbe802a410 comp rx=0 tx=0).stop 2024-04-23T18:35:25.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.920+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2487245886 shutdown_connections 2024-04-23T18:35:25.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.920+0000 7fdbfb57d640 1 --2- 172.21.15.132:0/2487245886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbf4102a00 0x7fdbf4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.920+0000 7fdbfb57d640 1 --2- 172.21.15.132:0/2487245886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf4102040 0x7fdbf41024c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.920+0000 7fdbfb57d640 1 --2- 172.21.15.132:0/2487245886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbf4100e60 0x7fdbf4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.920+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2487245886 >> 172.21.15.132:0/2487245886 conn(0x7fdbf40fc420 msgr2=0x7fdbf40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:25.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.920+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2487245886 shutdown_connections 2024-04-23T18:35:25.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.920+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2487245886 wait complete. 2024-04-23T18:35:25.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.920+0000 7fdbfb57d640 1 Processor -- start 2024-04-23T18:35:25.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.920+0000 7fdbfb57d640 1 -- start start 2024-04-23T18:35:25.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbfb57d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbf4100e60 0x7fdbf41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:25.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbfb57d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbf4102040 0x7fdbf4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbfb57d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf4102a00 0x7fdbf419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbfb57d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf410ca00 con 0x7fdbf4102a00 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbfb57d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf410c880 con 0x7fdbf4100e60 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbfb57d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf410cb80 con 0x7fdbf4102040 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf9b1a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf4102a00 0x7fdbf419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf9b1a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf4102a00 0x7fdbf419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49120/0 (socket says 172.21.15.132:49120) 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf9b1a640 1 -- 172.21.15.132:0/2126571153 learned_addr learned my addr 172.21.15.132:0/2126571153 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf8b18640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbf4102040 0x7fdbf4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf9319640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbf4100e60 0x7fdbf41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf9b1a640 1 -- 172.21.15.132:0/2126571153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbf4102040 msgr2=0x7fdbf4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf9b1a640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbf4102040 0x7fdbf4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf9b1a640 1 -- 172.21.15.132:0/2126571153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbf4100e60 msgr2=0x7fdbf41981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:25.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf9b1a640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbf4100e60 0x7fdbf41981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:25.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf9b1a640 1 -- 172.21.15.132:0/2126571153 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf419fdf0 con 0x7fdbf4102a00 2024-04-23T18:35:25.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf9319640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbf4100e60 0x7fdbf41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:25.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf9b1a640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf4102a00 0x7fdbf419f780 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400d750 tx=0x7fdbe400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:25.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf27fc640 1 -- 172.21.15.132:0/2126571153 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4017070 con 0x7fdbf4102a00 2024-04-23T18:35:25.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf27fc640 1 -- 172.21.15.132:0/2126571153 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe40054d0 con 0x7fdbf4102a00 2024-04-23T18:35:25.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbf27fc640 1 -- 172.21.15.132:0/2126571153 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4004c60 con 0x7fdbf4102a00 2024-04-23T18:35:25.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2126571153 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf41a00e0 con 0x7fdbf4102a00 2024-04-23T18:35:25.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.921+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2126571153 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf41a0600 con 0x7fdbf4102a00 2024-04-23T18:35:25.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.922+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2126571153 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf41a0860 con 0x7fdbf4102a00 2024-04-23T18:35:25.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.923+0000 7fdbf27fc640 1 -- 172.21.15.132:0/2126571153 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fdbe4009bd0 con 0x7fdbf4102a00 2024-04-23T18:35:25.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.924+0000 7fdbf27fc640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdbdc0a3bb0 0x7fdbdc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:25.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.924+0000 7fdbf9319640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdbdc0a3bb0 0x7fdbdc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:25.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.924+0000 7fdbf27fc640 1 -- 172.21.15.132:0/2126571153 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fdbe40cfd30 con 0x7fdbf4102a00 2024-04-23T18:35:25.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.924+0000 7fdbf9319640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdbdc0a3bb0 0x7fdbdc0a6070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4101bb0 tx=0x7fdbec009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:25.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:25.929+0000 7fdbf27fc640 1 -- 172.21.15.132:0/2126571153 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdbe409c700 con 0x7fdbf4102a00 2024-04-23T18:35:26.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.104+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2126571153 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdbf41a0a30 con 0x7fdbf4102a00 2024-04-23T18:35:26.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.104+0000 7fdbf27fc640 1 -- 172.21.15.132:0/2126571153 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fdbe409c000 con 0x7fdbf4102a00 2024-04-23T18:35:26.096 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-04-23T18:35:26.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.106+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2126571153 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdbdc0a3bb0 msgr2=0x7fdbdc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.106+0000 7fdbfb57d640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdbdc0a3bb0 0x7fdbdc0a6070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4101bb0 tx=0x7fdbec009530 comp rx=0 tx=0).stop 2024-04-23T18:35:26.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.106+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2126571153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf4102a00 msgr2=0x7fdbf419f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.106+0000 7fdbfb57d640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf4102a00 0x7fdbf419f780 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400d750 tx=0x7fdbe400dc10 comp rx=0 tx=0).stop 2024-04-23T18:35:26.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.106+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2126571153 shutdown_connections 2024-04-23T18:35:26.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.106+0000 7fdbfb57d640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdbdc0a3bb0 0x7fdbdc0a6070 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.106+0000 7fdbfb57d640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbf4102040 0x7fdbf4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.106+0000 7fdbfb57d640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf4102a00 0x7fdbf419f780 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.106+0000 7fdbfb57d640 1 --2- 172.21.15.132:0/2126571153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbf4100e60 0x7fdbf41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.107+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2126571153 >> 172.21.15.132:0/2126571153 conn(0x7fdbf40fc420 msgr2=0x7fdbf40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.107+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2126571153 shutdown_connections 2024-04-23T18:35:26.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.107+0000 7fdbfb57d640 1 -- 172.21.15.132:0/2126571153 wait complete. 2024-04-23T18:35:26.110 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:35:26.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.221+0000 7fd3a35af640 1 -- 172.21.15.132:0/1209682014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c100e70 msgr2=0x7fd39c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.221+0000 7fd3a35af640 1 --2- 172.21.15.132:0/1209682014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c100e70 0x7fd39c101290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd398009900 tx=0x7fd39802a410 comp rx=0 tx=0).stop 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.221+0000 7fd3a35af640 1 -- 172.21.15.132:0/1209682014 shutdown_connections 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.221+0000 7fd3a35af640 1 --2- 172.21.15.132:0/1209682014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd39c101fc0 0x7fd39c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.221+0000 7fd3a35af640 1 --2- 172.21.15.132:0/1209682014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd39c102980 0x7fd39c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.221+0000 7fd3a35af640 1 --2- 172.21.15.132:0/1209682014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c100e70 0x7fd39c101290 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.221+0000 7fd3a35af640 1 -- 172.21.15.132:0/1209682014 >> 172.21.15.132:0/1209682014 conn(0x7fd39c0fc3e0 msgr2=0x7fd39c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.221+0000 7fd3a35af640 1 -- 172.21.15.132:0/1209682014 shutdown_connections 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.221+0000 7fd3a35af640 1 -- 172.21.15.132:0/1209682014 wait complete. 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.222+0000 7fd3a35af640 1 Processor -- start 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.222+0000 7fd3a35af640 1 -- start start 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.222+0000 7fd3a35af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd39c100e70 0x7fd39c197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.222+0000 7fd3a35af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd39c101fc0 0x7fd39c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.222+0000 7fd3a35af640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c102980 0x7fd39c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.222+0000 7fd3a35af640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10c9a0 con 0x7fd39c102980 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.222+0000 7fd3a35af640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10c820 con 0x7fd39c100e70 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.222+0000 7fd3a35af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10cb20 con 0x7fd39c101fc0 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a1b4c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c102980 0x7fd39c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a1b4c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c102980 0x7fd39c19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49136/0 (socket says 172.21.15.132:49136) 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a1b4c640 1 -- 172.21.15.132:0/2499628210 learned_addr learned my addr 172.21.15.132:0/2499628210 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a0b4a640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd39c101fc0 0x7fd39c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a134b640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd39c100e70 0x7fd39c197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a1b4c640 1 -- 172.21.15.132:0/2499628210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd39c101fc0 msgr2=0x7fd39c1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a1b4c640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd39c101fc0 0x7fd39c1975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a1b4c640 1 -- 172.21.15.132:0/2499628210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd39c100e70 msgr2=0x7fd39c197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a1b4c640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd39c100e70 0x7fd39c197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a1b4c640 1 -- 172.21.15.132:0/2499628210 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c19ec90 con 0x7fd39c102980 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a1b4c640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c102980 0x7fd39c19e620 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd394007c30 tx=0x7fd39400e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3927fc640 1 -- 172.21.15.132:0/2499628210 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394019020 con 0x7fd39c102980 2024-04-23T18:35:26.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3927fc640 1 -- 172.21.15.132:0/2499628210 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd394007e40 con 0x7fd39c102980 2024-04-23T18:35:26.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a35af640 1 -- 172.21.15.132:0/2499628210 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39c069140 con 0x7fd39c102980 2024-04-23T18:35:26.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3927fc640 1 -- 172.21.15.132:0/2499628210 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394005560 con 0x7fd39c102980 2024-04-23T18:35:26.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.223+0000 7fd3a35af640 1 -- 172.21.15.132:0/2499628210 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39c069690 con 0x7fd39c102980 2024-04-23T18:35:26.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.225+0000 7fd3a35af640 1 -- 172.21.15.132:0/2499628210 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd39c068d70 con 0x7fd39c102980 2024-04-23T18:35:26.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.225+0000 7fd3927fc640 1 -- 172.21.15.132:0/2499628210 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fd39400eb60 con 0x7fd39c102980 2024-04-23T18:35:26.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.226+0000 7fd3927fc640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd3880a3c30 0x7fd3880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.226+0000 7fd3a134b640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd3880a3c30 0x7fd3880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.226+0000 7fd3927fc640 1 -- 172.21.15.132:0/2499628210 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fd3940d1c90 con 0x7fd39c102980 2024-04-23T18:35:26.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.227+0000 7fd3a134b640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd3880a3c30 0x7fd3880a60f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd398000c00 tx=0x7fd3980314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.230+0000 7fd3927fc640 1 -- 172.21.15.132:0/2499628210 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd394023020 con 0x7fd39c102980 2024-04-23T18:35:26.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:26 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2445488321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:26.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:26 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2126571153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:35:26.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.431+0000 7fd3a35af640 1 -- 172.21.15.132:0/2499628210 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd39c19ef20 con 0x7fd39c102980 2024-04-23T18:35:26.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.432+0000 7fd3927fc640 1 -- 172.21.15.132:0/2499628210 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fd39409e690 con 0x7fd39c102980 2024-04-23T18:35:26.424 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-04-23T18:35:26.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.434+0000 7fd3a35af640 1 -- 172.21.15.132:0/2499628210 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd3880a3c30 msgr2=0x7fd3880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.434+0000 7fd3a35af640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd3880a3c30 0x7fd3880a60f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd398000c00 tx=0x7fd3980314a0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.434+0000 7fd3a35af640 1 -- 172.21.15.132:0/2499628210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c102980 msgr2=0x7fd39c19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.434+0000 7fd3a35af640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c102980 0x7fd39c19e620 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd394007c30 tx=0x7fd39400e420 comp rx=0 tx=0).stop 2024-04-23T18:35:26.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.434+0000 7fd3a35af640 1 -- 172.21.15.132:0/2499628210 shutdown_connections 2024-04-23T18:35:26.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.434+0000 7fd3a35af640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fd3880a3c30 0x7fd3880a60f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.434+0000 7fd3a35af640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd39c101fc0 0x7fd39c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.434+0000 7fd3a35af640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c102980 0x7fd39c19e620 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.434+0000 7fd3a35af640 1 --2- 172.21.15.132:0/2499628210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd39c100e70 0x7fd39c197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.434+0000 7fd3a35af640 1 -- 172.21.15.132:0/2499628210 >> 172.21.15.132:0/2499628210 conn(0x7fd39c0fc3e0 msgr2=0x7fd39c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.434+0000 7fd3a35af640 1 -- 172.21.15.132:0/2499628210 shutdown_connections 2024-04-23T18:35:26.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.434+0000 7fd3a35af640 1 -- 172.21.15.132:0/2499628210 wait complete. 2024-04-23T18:35:26.438 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:35:26.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:26 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2445488321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:26.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:26 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2126571153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:35:26.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:26 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2445488321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:26.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:26 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2126571153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:35:26.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.551+0000 7f018502c640 1 -- 172.21.15.132:0/157343149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01801053e0 msgr2=0x7f01801077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.551+0000 7f018502c640 1 --2- 172.21.15.132:0/157343149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01801053e0 0x7f01801077d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0170009900 tx=0x7f017002a410 comp rx=0 tx=0).stop 2024-04-23T18:35:26.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.551+0000 7f018502c640 1 -- 172.21.15.132:0/157343149 shutdown_connections 2024-04-23T18:35:26.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.551+0000 7f018502c640 1 --2- 172.21.15.132:0/157343149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0180107d10 0x7f018010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.551+0000 7f018502c640 1 --2- 172.21.15.132:0/157343149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01801053e0 0x7f01801077d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.551+0000 7f018502c640 1 --2- 172.21.15.132:0/157343149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0180102ab0 0x7f0180104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.551+0000 7f018502c640 1 -- 172.21.15.132:0/157343149 >> 172.21.15.132:0/157343149 conn(0x7f01800fc400 msgr2=0x7f01800fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.551+0000 7f018502c640 1 -- 172.21.15.132:0/157343149 shutdown_connections 2024-04-23T18:35:26.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.551+0000 7f018502c640 1 -- 172.21.15.132:0/157343149 wait complete. 2024-04-23T18:35:26.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.551+0000 7f018502c640 1 Processor -- start 2024-04-23T18:35:26.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f018502c640 1 -- start start 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f018502c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0180102ab0 0x7f0180198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f018502c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01801053e0 0x7f01801987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f018502c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0180107d10 0x7f018019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f018502c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f018010daf0 con 0x7f01801053e0 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f018502c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f018010d970 con 0x7f0180107d10 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f018502c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f018010dc70 con 0x7f0180102ab0 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f017e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01801053e0 0x7f01801987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f017e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01801053e0 0x7f01801987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49162/0 (socket says 172.21.15.132:49162) 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f017e59c640 1 -- 172.21.15.132:0/2184634070 learned_addr learned my addr 172.21.15.132:0/2184634070 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f017ed9d640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0180102ab0 0x7f0180198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f017f59e640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0180107d10 0x7f018019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f017e59c640 1 -- 172.21.15.132:0/2184634070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0180102ab0 msgr2=0x7f0180198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f017e59c640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0180102ab0 0x7f0180198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f017e59c640 1 -- 172.21.15.132:0/2184634070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0180107d10 msgr2=0x7f018019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f017e59c640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0180107d10 0x7f018019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f017e59c640 1 -- 172.21.15.132:0/2184634070 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018019fe90 con 0x7f01801053e0 2024-04-23T18:35:26.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f017f59e640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0180107d10 0x7f018019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:26.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f017e59c640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01801053e0 0x7f01801987b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0170006010 tx=0x7f01700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.552+0000 7f0167fff640 1 -- 172.21.15.132:0/2184634070 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017003f030 con 0x7f01801053e0 2024-04-23T18:35:26.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.553+0000 7f018502c640 1 -- 172.21.15.132:0/2184634070 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01801a0120 con 0x7f01801053e0 2024-04-23T18:35:26.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.553+0000 7f0167fff640 1 -- 172.21.15.132:0/2184634070 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0170003cb0 con 0x7f01801053e0 2024-04-23T18:35:26.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.553+0000 7f0167fff640 1 -- 172.21.15.132:0/2184634070 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170035070 con 0x7f01801053e0 2024-04-23T18:35:26.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.553+0000 7f018502c640 1 -- 172.21.15.132:0/2184634070 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01801a0610 con 0x7f01801053e0 2024-04-23T18:35:26.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.554+0000 7f018502c640 1 -- 172.21.15.132:0/2184634070 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01800fdef0 con 0x7f01801053e0 2024-04-23T18:35:26.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.555+0000 7f0167fff640 1 -- 172.21.15.132:0/2184634070 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f0170042030 con 0x7f01801053e0 2024-04-23T18:35:26.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.555+0000 7f0167fff640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f01680a3bb0 0x7f01680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.556+0000 7f017ed9d640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f01680a3bb0 0x7f01680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.556+0000 7f0167fff640 1 -- 172.21.15.132:0/2184634070 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f01700f0b00 con 0x7f01801053e0 2024-04-23T18:35:26.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.556+0000 7f017ed9d640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f01680a3bb0 0x7f01680a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0174005fd0 tx=0x7f0174005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.560+0000 7f0167fff640 1 -- 172.21.15.132:0/2184634070 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01700bd4d0 con 0x7f01801053e0 2024-04-23T18:35:26.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.747+0000 7f018502c640 1 -- 172.21.15.132:0/2184634070 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0180062b50 con 0x7f01801053e0 2024-04-23T18:35:26.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.748+0000 7f0167fff640 1 -- 172.21.15.132:0/2184634070 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f01700bcdd0 con 0x7f01801053e0 2024-04-23T18:35:26.740 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-04-23T18:35:26.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.750+0000 7f018502c640 1 -- 172.21.15.132:0/2184634070 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f01680a3bb0 msgr2=0x7f01680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.750+0000 7f018502c640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f01680a3bb0 0x7f01680a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0174005fd0 tx=0x7f0174005850 comp rx=0 tx=0).stop 2024-04-23T18:35:26.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.750+0000 7f018502c640 1 -- 172.21.15.132:0/2184634070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01801053e0 msgr2=0x7f01801987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.750+0000 7f018502c640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01801053e0 0x7f01801987b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0170006010 tx=0x7f01700042f0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.750+0000 7f018502c640 1 -- 172.21.15.132:0/2184634070 shutdown_connections 2024-04-23T18:35:26.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.750+0000 7f018502c640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f01680a3bb0 0x7f01680a6070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.750+0000 7f018502c640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0180107d10 0x7f018019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.750+0000 7f018502c640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01801053e0 0x7f01801987b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.750+0000 7f018502c640 1 --2- 172.21.15.132:0/2184634070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0180102ab0 0x7f0180198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.750+0000 7f018502c640 1 -- 172.21.15.132:0/2184634070 >> 172.21.15.132:0/2184634070 conn(0x7f01800fc400 msgr2=0x7f0180104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.750+0000 7f018502c640 1 -- 172.21.15.132:0/2184634070 shutdown_connections 2024-04-23T18:35:26.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.750+0000 7f018502c640 1 -- 172.21.15.132:0/2184634070 wait complete. 2024-04-23T18:35:26.754 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:35:26.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.862+0000 7ff897194640 1 -- 172.21.15.132:0/1717153814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff890101fb0 msgr2=0x7ff8901023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.862+0000 7ff897194640 1 --2- 172.21.15.132:0/1717153814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff890101fb0 0x7ff8901023d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff884009900 tx=0x7ff88402a410 comp rx=0 tx=0).stop 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.862+0000 7ff897194640 1 -- 172.21.15.132:0/1717153814 shutdown_connections 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.862+0000 7ff897194640 1 --2- 172.21.15.132:0/1717153814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8901031a0 0x7ff890103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.862+0000 7ff897194640 1 --2- 172.21.15.132:0/1717153814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff890103b60 0x7ff89010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.862+0000 7ff897194640 1 --2- 172.21.15.132:0/1717153814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff890101fb0 0x7ff8901023d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.862+0000 7ff897194640 1 -- 172.21.15.132:0/1717153814 >> 172.21.15.132:0/1717153814 conn(0x7ff8900fd500 msgr2=0x7ff8900ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.862+0000 7ff897194640 1 -- 172.21.15.132:0/1717153814 shutdown_connections 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.862+0000 7ff897194640 1 -- 172.21.15.132:0/1717153814 wait complete. 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff897194640 1 Processor -- start 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff897194640 1 -- start start 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff897194640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff890101fb0 0x7ff890198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff897194640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8901031a0 0x7ff890198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff897194640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff890103b60 0x7ff89019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff897194640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff89010daf0 con 0x7ff890101fb0 2024-04-23T18:35:26.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff897194640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff89010d970 con 0x7ff890103b60 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff897194640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff89010dc70 con 0x7ff8901031a0 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff894f30640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff890101fb0 0x7ff890198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff894f30640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff890101fb0 0x7ff890198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49178/0 (socket says 172.21.15.132:49178) 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff894f30640 1 -- 172.21.15.132:0/708340911 learned_addr learned my addr 172.21.15.132:0/708340911 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff88ffff640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8901031a0 0x7ff890198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff895731640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff890103b60 0x7ff89019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff894f30640 1 -- 172.21.15.132:0/708340911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8901031a0 msgr2=0x7ff890198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff894f30640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8901031a0 0x7ff890198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff894f30640 1 -- 172.21.15.132:0/708340911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff890103b60 msgr2=0x7ff89019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff894f30640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff890103b60 0x7ff89019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff894f30640 1 -- 172.21.15.132:0/708340911 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89019fe70 con 0x7ff890101fb0 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.863+0000 7ff895731640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff890103b60 0x7ff89019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:26.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.864+0000 7ff894f30640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff890101fb0 0x7ff890198250 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff884006010 tx=0x7ff884004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.864+0000 7ff88dffb640 1 -- 172.21.15.132:0/708340911 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88403f030 con 0x7ff890101fb0 2024-04-23T18:35:26.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.864+0000 7ff897194640 1 -- 172.21.15.132:0/708340911 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8901a0100 con 0x7ff890101fb0 2024-04-23T18:35:26.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.864+0000 7ff88dffb640 1 -- 172.21.15.132:0/708340911 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff884003cb0 con 0x7ff890101fb0 2024-04-23T18:35:26.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.864+0000 7ff88dffb640 1 -- 172.21.15.132:0/708340911 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884035070 con 0x7ff890101fb0 2024-04-23T18:35:26.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.864+0000 7ff897194640 1 -- 172.21.15.132:0/708340911 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8901a0640 con 0x7ff890101fb0 2024-04-23T18:35:26.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.865+0000 7ff897194640 1 -- 172.21.15.132:0/708340911 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8901093c0 con 0x7ff890101fb0 2024-04-23T18:35:26.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.866+0000 7ff88dffb640 1 -- 172.21.15.132:0/708340911 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7ff884042030 con 0x7ff890101fb0 2024-04-23T18:35:26.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.867+0000 7ff88dffb640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff8800a3bb0 0x7ff8800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:26.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.867+0000 7ff88ffff640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff8800a3bb0 0x7ff8800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:26.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.867+0000 7ff88dffb640 1 -- 172.21.15.132:0/708340911 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff8840f0fb0 con 0x7ff890101fb0 2024-04-23T18:35:26.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.868+0000 7ff88ffff640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff8800a3bb0 0x7ff8800a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff87c005fd0 tx=0x7ff87c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:26.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:26.871+0000 7ff88dffb640 1 -- 172.21.15.132:0/708340911 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8840bd980 con 0x7ff890101fb0 2024-04-23T18:35:27.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.094+0000 7ff897194640 1 -- 172.21.15.132:0/708340911 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff89002d5c0 con 0x7ff890101fb0 2024-04-23T18:35:27.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.095+0000 7ff88dffb640 1 -- 172.21.15.132:0/708340911 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11946 (secure 0 0 0) 0x7ff884003e60 con 0x7ff890101fb0 2024-04-23T18:35:27.087 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:27.087 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:34:07.536766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24394,24425,24460]},"id":1}]} 2024-04-23T18:35:27.088 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 25 2024-04-23T18:35:27.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.097+0000 7ff897194640 1 -- 172.21.15.132:0/708340911 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff8800a3bb0 msgr2=0x7ff8800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.097+0000 7ff897194640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff8800a3bb0 0x7ff8800a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff87c005fd0 tx=0x7ff87c005e20 comp rx=0 tx=0).stop 2024-04-23T18:35:27.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.097+0000 7ff897194640 1 -- 172.21.15.132:0/708340911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff890101fb0 msgr2=0x7ff890198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.097+0000 7ff897194640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff890101fb0 0x7ff890198250 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff884006010 tx=0x7ff884004430 comp rx=0 tx=0).stop 2024-04-23T18:35:27.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.097+0000 7ff897194640 1 -- 172.21.15.132:0/708340911 shutdown_connections 2024-04-23T18:35:27.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.097+0000 7ff897194640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7ff8800a3bb0 0x7ff8800a6070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.097+0000 7ff897194640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8901031a0 0x7ff890198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.097+0000 7ff897194640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff890103b60 0x7ff89019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.097+0000 7ff897194640 1 --2- 172.21.15.132:0/708340911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff890101fb0 0x7ff890198250 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.097+0000 7ff897194640 1 -- 172.21.15.132:0/708340911 >> 172.21.15.132:0/708340911 conn(0x7ff8900fd500 msgr2=0x7ff890106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:27.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.097+0000 7ff897194640 1 -- 172.21.15.132:0/708340911 shutdown_connections 2024-04-23T18:35:27.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.097+0000 7ff897194640 1 -- 172.21.15.132:0/708340911 wait complete. 2024-04-23T18:35:27.100 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:35:27.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.208+0000 7f5b46b6f640 1 -- 172.21.15.132:0/380385114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b40100e90 msgr2=0x7f5b401012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.208+0000 7f5b46b6f640 1 --2- 172.21.15.132:0/380385114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b40100e90 0x7f5b401012b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009900 tx=0x7f5b3402a3f0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.208+0000 7f5b46b6f640 1 -- 172.21.15.132:0/380385114 shutdown_connections 2024-04-23T18:35:27.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.208+0000 7f5b46b6f640 1 --2- 172.21.15.132:0/380385114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b401029a0 0x7f5b401092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.208+0000 7f5b46b6f640 1 --2- 172.21.15.132:0/380385114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b40101fe0 0x7f5b40102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.208+0000 7f5b46b6f640 1 --2- 172.21.15.132:0/380385114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b40100e90 0x7f5b401012b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.208+0000 7f5b46b6f640 1 -- 172.21.15.132:0/380385114 >> 172.21.15.132:0/380385114 conn(0x7f5b400fc400 msgr2=0x7f5b400fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.208+0000 7f5b46b6f640 1 -- 172.21.15.132:0/380385114 shutdown_connections 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.208+0000 7f5b46b6f640 1 -- 172.21.15.132:0/380385114 wait complete. 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b46b6f640 1 Processor -- start 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b46b6f640 1 -- start start 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b46b6f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b40100e90 0x7f5b40196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b46b6f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b40101fe0 0x7f5b40197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b46b6f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b401029a0 0x7f5b4019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b46b6f640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4010ca10 con 0x7f5b40100e90 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b46b6f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4010c890 con 0x7f5b401029a0 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b46b6f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4010cb90 con 0x7f5b40101fe0 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b4490b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b40100e90 0x7f5b40196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b4490b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b40100e90 0x7f5b40196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49200/0 (socket says 172.21.15.132:49200) 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b4490b640 1 -- 172.21.15.132:0/2241307076 learned_addr learned my addr 172.21.15.132:0/2241307076 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b4510c640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b401029a0 0x7f5b4019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:27.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b4490b640 1 -- 172.21.15.132:0/2241307076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b40101fe0 msgr2=0x7f5b40197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b4490b640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b40101fe0 0x7f5b40197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b4490b640 1 -- 172.21.15.132:0/2241307076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b401029a0 msgr2=0x7f5b4019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b4490b640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b401029a0 0x7f5b4019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.209+0000 7f5b4490b640 1 -- 172.21.15.132:0/2241307076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4019ec10 con 0x7f5b40100e90 2024-04-23T18:35:27.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.210+0000 7f5b4490b640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b40100e90 0x7f5b40196ff0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5b34031500 tx=0x7f5b34003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:27.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.210+0000 7f5b3dffb640 1 -- 172.21.15.132:0/2241307076 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3403f070 con 0x7f5b40100e90 2024-04-23T18:35:27.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.210+0000 7f5b3dffb640 1 -- 172.21.15.132:0/2241307076 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b3403b460 con 0x7f5b40100e90 2024-04-23T18:35:27.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.210+0000 7f5b46b6f640 1 -- 172.21.15.132:0/2241307076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4019eea0 con 0x7f5b40100e90 2024-04-23T18:35:27.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.210+0000 7f5b3dffb640 1 -- 172.21.15.132:0/2241307076 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34035030 con 0x7f5b40100e90 2024-04-23T18:35:27.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.210+0000 7f5b46b6f640 1 -- 172.21.15.132:0/2241307076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4019f300 con 0x7f5b40100e90 2024-04-23T18:35:27.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.211+0000 7f5b46b6f640 1 -- 172.21.15.132:0/2241307076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b40108140 con 0x7f5b40100e90 2024-04-23T18:35:27.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.213+0000 7f5b3dffb640 1 -- 172.21.15.132:0/2241307076 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f5b34042070 con 0x7f5b40100e90 2024-04-23T18:35:27.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.213+0000 7f5b3dffb640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5b300a3bb0 0x7f5b300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:27.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.213+0000 7f5b3ffff640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5b300a3bb0 0x7f5b300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:27.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.213+0000 7f5b3dffb640 1 -- 172.21.15.132:0/2241307076 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5b340f1260 con 0x7f5b40100e90 2024-04-23T18:35:27.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.214+0000 7f5b3ffff640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5b300a3bb0 0x7f5b300a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5b40198500 tx=0x7f5b2c005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:27.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.217+0000 7f5b3dffb640 1 -- 172.21.15.132:0/2241307076 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b340bdc30 con 0x7f5b40100e90 2024-04-23T18:35:27.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:27 smithi132 ceph-mon[29178]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-23T18:35:27.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:27 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2499628210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:35:27.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:27 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2184634070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:35:27.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:27 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/708340911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:27.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.403+0000 7f5b46b6f640 1 -- 172.21.15.132:0/2241307076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b4002d810 con 0x7f5b40100e90 2024-04-23T18:35:27.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.405+0000 7f5b3dffb640 1 -- 172.21.15.132:0/2241307076 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22800 (secure 0 0 0) 0x7f5b340bd530 con 0x7f5b40100e90 2024-04-23T18:35:27.397 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:27.397 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":88,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:34:55.248852+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:35:27.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.407+0000 7f5b46b6f640 1 -- 172.21.15.132:0/2241307076 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5b300a3bb0 msgr2=0x7f5b300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.407+0000 7f5b46b6f640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5b300a3bb0 0x7f5b300a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5b40198500 tx=0x7f5b2c005c90 comp rx=0 tx=0).stop 2024-04-23T18:35:27.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.407+0000 7f5b46b6f640 1 -- 172.21.15.132:0/2241307076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b40100e90 msgr2=0x7f5b40196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.407+0000 7f5b46b6f640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b40100e90 0x7f5b40196ff0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5b34031500 tx=0x7f5b34003d50 comp rx=0 tx=0).stop 2024-04-23T18:35:27.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.407+0000 7f5b46b6f640 1 -- 172.21.15.132:0/2241307076 shutdown_connections 2024-04-23T18:35:27.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.407+0000 7f5b46b6f640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5b300a3bb0 0x7f5b300a6070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.407+0000 7f5b46b6f640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b40101fe0 0x7f5b40197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.407+0000 7f5b46b6f640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b401029a0 0x7f5b4019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.407+0000 7f5b46b6f640 1 --2- 172.21.15.132:0/2241307076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b40100e90 0x7f5b40196ff0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.407+0000 7f5b46b6f640 1 -- 172.21.15.132:0/2241307076 >> 172.21.15.132:0/2241307076 conn(0x7f5b400fc400 msgr2=0x7f5b40105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:27.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.407+0000 7f5b46b6f640 1 -- 172.21.15.132:0/2241307076 shutdown_connections 2024-04-23T18:35:27.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.407+0000 7f5b46b6f640 1 -- 172.21.15.132:0/2241307076 wait complete. 2024-04-23T18:35:27.411 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:29:20.132451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:33:58.407521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:33:59.518206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:35:27.411 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:35:27.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.520+0000 7f3b28dfe640 1 -- 172.21.15.132:0/464505752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b24100e90 msgr2=0x7f3b241012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.520+0000 7f3b28dfe640 1 --2- 172.21.15.132:0/464505752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b24100e90 0x7f3b241012b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009ca0 tx=0x7f3b1402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.520+0000 7f3b28dfe640 1 -- 172.21.15.132:0/464505752 shutdown_connections 2024-04-23T18:35:27.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.520+0000 7f3b28dfe640 1 --2- 172.21.15.132:0/464505752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b24101fe0 0x7f3b24102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.520+0000 7f3b28dfe640 1 --2- 172.21.15.132:0/464505752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b241029a0 0x7f3b241092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.520+0000 7f3b28dfe640 1 --2- 172.21.15.132:0/464505752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b24100e90 0x7f3b241012b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.520+0000 7f3b28dfe640 1 -- 172.21.15.132:0/464505752 >> 172.21.15.132:0/464505752 conn(0x7f3b240fc400 msgr2=0x7f3b240fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:27.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.520+0000 7f3b28dfe640 1 -- 172.21.15.132:0/464505752 shutdown_connections 2024-04-23T18:35:27.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.520+0000 7f3b28dfe640 1 -- 172.21.15.132:0/464505752 wait complete. 2024-04-23T18:35:27.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b28dfe640 1 Processor -- start 2024-04-23T18:35:27.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b28dfe640 1 -- start start 2024-04-23T18:35:27.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b28dfe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b24100e90 0x7f3b241981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:27.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b28dfe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b24101fe0 0x7f3b24198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:27.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b28dfe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b241029a0 0x7f3b2419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:27.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b28dfe640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2410daf0 con 0x7f3b24101fe0 2024-04-23T18:35:27.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b28dfe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2410d970 con 0x7f3b241029a0 2024-04-23T18:35:27.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b28dfe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2410dc70 con 0x7f3b24100e90 2024-04-23T18:35:27.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b21d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b24101fe0 0x7f3b24198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:27.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b21d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b24101fe0 0x7f3b24198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49226/0 (socket says 172.21.15.132:49226) 2024-04-23T18:35:27.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b21d9b640 1 -- 172.21.15.132:0/2340215281 learned_addr learned my addr 172.21.15.132:0/2340215281 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:27.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b22d9d640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b241029a0 0x7f3b2419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:27.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b2259c640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b24100e90 0x7f3b241981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:27.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b21d9b640 1 -- 172.21.15.132:0/2340215281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b24100e90 msgr2=0x7f3b241981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b21d9b640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b24100e90 0x7f3b241981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b21d9b640 1 -- 172.21.15.132:0/2340215281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b241029a0 msgr2=0x7f3b2419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.521+0000 7f3b21d9b640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b241029a0 0x7f3b2419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.522+0000 7f3b21d9b640 1 -- 172.21.15.132:0/2340215281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2419fdf0 con 0x7f3b24101fe0 2024-04-23T18:35:27.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.522+0000 7f3b22d9d640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b241029a0 0x7f3b2419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:27.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.522+0000 7f3b21d9b640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b24101fe0 0x7f3b24198710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800d950 tx=0x7f3b1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:27.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.522+0000 7f3b137fe640 1 -- 172.21.15.132:0/2340215281 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18017020 con 0x7f3b24101fe0 2024-04-23T18:35:27.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.522+0000 7f3b137fe640 1 -- 172.21.15.132:0/2340215281 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b18009c30 con 0x7f3b24101fe0 2024-04-23T18:35:27.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.522+0000 7f3b28dfe640 1 -- 172.21.15.132:0/2340215281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b241a00e0 con 0x7f3b24101fe0 2024-04-23T18:35:27.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.522+0000 7f3b137fe640 1 -- 172.21.15.132:0/2340215281 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800f6b0 con 0x7f3b24101fe0 2024-04-23T18:35:27.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.522+0000 7f3b28dfe640 1 -- 172.21.15.132:0/2340215281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b241a0630 con 0x7f3b24101fe0 2024-04-23T18:35:27.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.523+0000 7f3b28dfe640 1 -- 172.21.15.132:0/2340215281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b24108210 con 0x7f3b24101fe0 2024-04-23T18:35:27.519 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:27 smithi148 ceph-mon[32311]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-23T18:35:27.519 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:27 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2499628210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:35:27.519 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:27 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2184634070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:35:27.519 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:27 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/708340911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:27.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.524+0000 7f3b137fe640 1 -- 172.21.15.132:0/2340215281 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f3b1801a020 con 0x7f3b24101fe0 2024-04-23T18:35:27.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.525+0000 7f3b137fe640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3b0c0a3bb0 0x7f3b0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:27.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.525+0000 7f3b2259c640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3b0c0a3bb0 0x7f3b0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:27.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.525+0000 7f3b137fe640 1 -- 172.21.15.132:0/2340215281 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f3b180d0190 con 0x7f3b24101fe0 2024-04-23T18:35:27.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.525+0000 7f3b2259c640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3b0c0a3bb0 0x7f3b0c0a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3b14031b50 tx=0x7f3b14005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:27.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.529+0000 7f3b137fe640 1 -- 172.21.15.132:0/2340215281 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b18009de0 con 0x7f3b24101fe0 2024-04-23T18:35:27.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:27 smithi157 ceph-mon[32580]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-23T18:35:27.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:27 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2499628210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:35:27.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:27 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2184634070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:35:27.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:27 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/708340911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:27.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.710+0000 7f3b28dfe640 1 -- 172.21.15.132:0/2340215281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3b24062b50 con 0x7f3b24101fe0 2024-04-23T18:35:27.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.710+0000 7f3b137fe640 1 -- 172.21.15.132:0/2340215281 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f3b1809cb60 con 0x7f3b24101fe0 2024-04-23T18:35:27.703 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-04-23T18:35:27.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.712+0000 7f3b28dfe640 1 -- 172.21.15.132:0/2340215281 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3b0c0a3bb0 msgr2=0x7f3b0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.712+0000 7f3b28dfe640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3b0c0a3bb0 0x7f3b0c0a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3b14031b50 tx=0x7f3b14005ef0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.712+0000 7f3b28dfe640 1 -- 172.21.15.132:0/2340215281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b24101fe0 msgr2=0x7f3b24198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.712+0000 7f3b28dfe640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b24101fe0 0x7f3b24198710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800d950 tx=0x7f3b1800de10 comp rx=0 tx=0).stop 2024-04-23T18:35:27.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.712+0000 7f3b28dfe640 1 -- 172.21.15.132:0/2340215281 shutdown_connections 2024-04-23T18:35:27.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.712+0000 7f3b28dfe640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f3b0c0a3bb0 0x7f3b0c0a6070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.712+0000 7f3b28dfe640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b241029a0 0x7f3b2419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.712+0000 7f3b28dfe640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b24101fe0 0x7f3b24198710 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.712+0000 7f3b28dfe640 1 --2- 172.21.15.132:0/2340215281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b24100e90 0x7f3b241981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.712+0000 7f3b28dfe640 1 -- 172.21.15.132:0/2340215281 >> 172.21.15.132:0/2340215281 conn(0x7f3b240fc400 msgr2=0x7f3b241053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:27.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.712+0000 7f3b28dfe640 1 -- 172.21.15.132:0/2340215281 shutdown_connections 2024-04-23T18:35:27.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.712+0000 7f3b28dfe640 1 -- 172.21.15.132:0/2340215281 wait complete. 2024-04-23T18:35:27.716 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:35:27.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.824+0000 7fe2afbf7640 1 -- 172.21.15.132:0/3420218367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a8102980 msgr2=0x7fe2a81092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.824+0000 7fe2afbf7640 1 --2- 172.21.15.132:0/3420218367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a8102980 0x7fe2a81092a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009ca0 tx=0x7fe2a002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.824+0000 7fe2afbf7640 1 -- 172.21.15.132:0/3420218367 shutdown_connections 2024-04-23T18:35:27.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.824+0000 7fe2afbf7640 1 --2- 172.21.15.132:0/3420218367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a8102980 0x7fe2a81092a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.824+0000 7fe2afbf7640 1 --2- 172.21.15.132:0/3420218367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2a8101fc0 0x7fe2a8102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.824+0000 7fe2afbf7640 1 --2- 172.21.15.132:0/3420218367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a8100e70 0x7fe2a8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.824+0000 7fe2afbf7640 1 -- 172.21.15.132:0/3420218367 >> 172.21.15.132:0/3420218367 conn(0x7fe2a80fc3e0 msgr2=0x7fe2a80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:27.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.824+0000 7fe2afbf7640 1 -- 172.21.15.132:0/3420218367 shutdown_connections 2024-04-23T18:35:27.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2afbf7640 1 -- 172.21.15.132:0/3420218367 wait complete. 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2afbf7640 1 Processor -- start 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2afbf7640 1 -- start start 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2afbf7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2a8100e70 0x7fe2a81970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2afbf7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a8101fc0 0x7fe2a8197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2afbf7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a8102980 0x7fe2a819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2afbf7640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a810ca60 con 0x7fe2a8101fc0 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2afbf7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a810c8e0 con 0x7fe2a8100e70 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2afbf7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a810cbe0 con 0x7fe2a8102980 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2ad192640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a8101fc0 0x7fe2a8197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2ad192640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a8101fc0 0x7fe2a8197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49248/0 (socket says 172.21.15.132:49248) 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2ad192640 1 -- 172.21.15.132:0/1279751848 learned_addr learned my addr 172.21.15.132:0/1279751848 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.825+0000 7fe2ad993640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2a8100e70 0x7fe2a81970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.826+0000 7fe2ae194640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a8102980 0x7fe2a819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:27.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.826+0000 7fe2ad192640 1 -- 172.21.15.132:0/1279751848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a8102980 msgr2=0x7fe2a819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.826+0000 7fe2ad192640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a8102980 0x7fe2a819e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.826+0000 7fe2ad192640 1 -- 172.21.15.132:0/1279751848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2a8100e70 msgr2=0x7fe2a81970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:27.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.826+0000 7fe2ad192640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2a8100e70 0x7fe2a81970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:27.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.826+0000 7fe2ad192640 1 -- 172.21.15.132:0/1279751848 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a819ecf0 con 0x7fe2a8101fc0 2024-04-23T18:35:27.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.826+0000 7fe2ad192640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a8101fc0 0x7fe2a8197610 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe29800b450 tx=0x7fe29800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:27.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.826+0000 7fe29effd640 1 -- 172.21.15.132:0/1279751848 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298015020 con 0x7fe2a8101fc0 2024-04-23T18:35:27.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.826+0000 7fe29effd640 1 -- 172.21.15.132:0/1279751848 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe298011d40 con 0x7fe2a8101fc0 2024-04-23T18:35:27.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.826+0000 7fe2afbf7640 1 -- 172.21.15.132:0/1279751848 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a819efe0 con 0x7fe2a8101fc0 2024-04-23T18:35:27.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.826+0000 7fe29effd640 1 -- 172.21.15.132:0/1279751848 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29800f340 con 0x7fe2a8101fc0 2024-04-23T18:35:27.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.826+0000 7fe2afbf7640 1 -- 172.21.15.132:0/1279751848 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a819f4c0 con 0x7fe2a8101fc0 2024-04-23T18:35:27.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.827+0000 7fe2afbf7640 1 -- 172.21.15.132:0/1279751848 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a81081d0 con 0x7fe2a8101fc0 2024-04-23T18:35:27.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.828+0000 7fe29effd640 1 -- 172.21.15.132:0/1279751848 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fe298018020 con 0x7fe2a8101fc0 2024-04-23T18:35:27.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.829+0000 7fe29effd640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe2940a3bb0 0x7fe2940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:27.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.829+0000 7fe2ad993640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe2940a3bb0 0x7fe2940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:27.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.829+0000 7fe29effd640 1 -- 172.21.15.132:0/1279751848 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe2980d04b0 con 0x7fe2a8101fc0 2024-04-23T18:35:27.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.829+0000 7fe2ad993640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe2940a3bb0 0x7fe2940a6070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4005fd0 tx=0x7fe2a4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:27.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:27.833+0000 7fe29effd640 1 -- 172.21.15.132:0/1279751848 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe29801f020 con 0x7fe2a8101fc0 2024-04-23T18:35:28.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.008+0000 7fe2afbf7640 1 -- 172.21.15.132:0/1279751848 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe2a802d810 con 0x7fe2a8101fc0 2024-04-23T18:35:28.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.008+0000 7fe29effd640 1 -- 172.21.15.132:0/1279751848 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fe29800d040 con 0x7fe2a8101fc0 2024-04-23T18:35:28.000 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-04-23T18:35:28.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.010+0000 7fe2afbf7640 1 -- 172.21.15.132:0/1279751848 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe2940a3bb0 msgr2=0x7fe2940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.010+0000 7fe2afbf7640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe2940a3bb0 0x7fe2940a6070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4005fd0 tx=0x7fe2a4005850 comp rx=0 tx=0).stop 2024-04-23T18:35:28.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.010+0000 7fe2afbf7640 1 -- 172.21.15.132:0/1279751848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a8101fc0 msgr2=0x7fe2a8197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.010+0000 7fe2afbf7640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a8101fc0 0x7fe2a8197610 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe29800b450 tx=0x7fe29800b910 comp rx=0 tx=0).stop 2024-04-23T18:35:28.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.010+0000 7fe2afbf7640 1 -- 172.21.15.132:0/1279751848 shutdown_connections 2024-04-23T18:35:28.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.010+0000 7fe2afbf7640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe2940a3bb0 0x7fe2940a6070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.004 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.010+0000 7fe2afbf7640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a8102980 0x7fe2a819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.004 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.010+0000 7fe2afbf7640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a8101fc0 0x7fe2a8197610 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.004 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.010+0000 7fe2afbf7640 1 --2- 172.21.15.132:0/1279751848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2a8100e70 0x7fe2a81970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.004 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.010+0000 7fe2afbf7640 1 -- 172.21.15.132:0/1279751848 >> 172.21.15.132:0/1279751848 conn(0x7fe2a80fc3e0 msgr2=0x7fe2a81053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:28.004 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.010+0000 7fe2afbf7640 1 -- 172.21.15.132:0/1279751848 shutdown_connections 2024-04-23T18:35:28.004 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.010+0000 7fe2afbf7640 1 -- 172.21.15.132:0/1279751848 wait complete. 2024-04-23T18:35:28.014 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:35:28.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.122+0000 7f48f59d8640 1 -- 172.21.15.132:0/742594492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48f0101fe0 msgr2=0x7f48f0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.122+0000 7f48f59d8640 1 --2- 172.21.15.132:0/742594492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48f0101fe0 0x7f48f0102460 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009900 tx=0x7f48e402a410 comp rx=0 tx=0).stop 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.122+0000 7f48f59d8640 1 -- 172.21.15.132:0/742594492 shutdown_connections 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.122+0000 7f48f59d8640 1 --2- 172.21.15.132:0/742594492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48f01029a0 0x7f48f01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.122+0000 7f48f59d8640 1 --2- 172.21.15.132:0/742594492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48f0101fe0 0x7f48f0102460 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.122+0000 7f48f59d8640 1 --2- 172.21.15.132:0/742594492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f0100e90 0x7f48f01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.122+0000 7f48f59d8640 1 -- 172.21.15.132:0/742594492 >> 172.21.15.132:0/742594492 conn(0x7f48f00fc400 msgr2=0x7f48f00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.122+0000 7f48f59d8640 1 -- 172.21.15.132:0/742594492 shutdown_connections 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.122+0000 7f48f59d8640 1 -- 172.21.15.132:0/742594492 wait complete. 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48f59d8640 1 Processor -- start 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48f59d8640 1 -- start start 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48f59d8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48f0100e90 0x7f48f0069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48f59d8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f0101fe0 0x7f48f019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48f59d8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48f01029a0 0x7f48f019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48f59d8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f48f010c9e0 con 0x7f48f0100e90 2024-04-23T18:35:28.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48f59d8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f48f010c860 con 0x7f48f0101fe0 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48f59d8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f48f010cb60 con 0x7f48f01029a0 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48eeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48f0100e90 0x7f48f0069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48eeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48f0100e90 0x7f48f0069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49262/0 (socket says 172.21.15.132:49262) 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48eeffd640 1 -- 172.21.15.132:0/1685864986 learned_addr learned my addr 172.21.15.132:0/1685864986 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48ee7fc640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f0101fe0 0x7f48f019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.123+0000 7f48ef7fe640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48f01029a0 0x7f48f019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.124+0000 7f48eeffd640 1 -- 172.21.15.132:0/1685864986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48f01029a0 msgr2=0x7f48f019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.124+0000 7f48eeffd640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48f01029a0 0x7f48f019f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.124+0000 7f48eeffd640 1 -- 172.21.15.132:0/1685864986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f0101fe0 msgr2=0x7f48f019ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.124+0000 7f48eeffd640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f0101fe0 0x7f48f019ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.124+0000 7f48eeffd640 1 -- 172.21.15.132:0/1685864986 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f019fdb0 con 0x7f48f0100e90 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.124+0000 7f48ef7fe640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48f01029a0 0x7f48f019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.124+0000 7f48eeffd640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48f0100e90 0x7f48f0069c70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f48e000cce0 tx=0x7f48e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:28.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.124+0000 7f48f49d6640 1 -- 172.21.15.132:0/1685864986 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e000f650 con 0x7f48f0100e90 2024-04-23T18:35:28.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.124+0000 7f48f59d8640 1 -- 172.21.15.132:0/1685864986 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f01a00a0 con 0x7f48f0100e90 2024-04-23T18:35:28.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.124+0000 7f48f49d6640 1 -- 172.21.15.132:0/1685864986 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48e000fca0 con 0x7f48f0100e90 2024-04-23T18:35:28.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.124+0000 7f48f49d6640 1 -- 172.21.15.132:0/1685864986 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0018b40 con 0x7f48f0100e90 2024-04-23T18:35:28.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.124+0000 7f48f59d8640 1 -- 172.21.15.132:0/1685864986 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f01a05c0 con 0x7f48f0100e90 2024-04-23T18:35:28.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.125+0000 7f48f59d8640 1 -- 172.21.15.132:0/1685864986 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f01081c0 con 0x7f48f0100e90 2024-04-23T18:35:28.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.126+0000 7f48f49d6640 1 -- 172.21.15.132:0/1685864986 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f48e000f7f0 con 0x7f48f0100e90 2024-04-23T18:35:28.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.127+0000 7f48f49d6640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f48d80a3bb0 0x7f48d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:28.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.127+0000 7f48ee7fc640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f48d80a3bb0 0x7f48d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:28.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.127+0000 7f48f49d6640 1 -- 172.21.15.132:0/1685864986 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f48e00d0990 con 0x7f48f0100e90 2024-04-23T18:35:28.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.127+0000 7f48ee7fc640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f48d80a3bb0 0x7f48d80a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f48e4000c00 tx=0x7f48e40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:28.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.131+0000 7f48f49d6640 1 -- 172.21.15.132:0/1685864986 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48e0014030 con 0x7f48f0100e90 2024-04-23T18:35:28.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.313+0000 7f48f59d8640 1 -- 172.21.15.132:0/1685864986 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f48f0062b50 con 0x7f48f0100e90 2024-04-23T18:35:28.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.313+0000 7f48f49d6640 1 -- 172.21.15.132:0/1685864986 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f48e00bbe20 con 0x7f48f0100e90 2024-04-23T18:35:28.306 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-04-23T18:35:28.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.315+0000 7f48f59d8640 1 -- 172.21.15.132:0/1685864986 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f48d80a3bb0 msgr2=0x7f48d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.315+0000 7f48f59d8640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f48d80a3bb0 0x7f48d80a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f48e4000c00 tx=0x7f48e40314a0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.315+0000 7f48f59d8640 1 -- 172.21.15.132:0/1685864986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48f0100e90 msgr2=0x7f48f0069c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.315+0000 7f48f59d8640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48f0100e90 0x7f48f0069c70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f48e000cce0 tx=0x7f48e000a620 comp rx=0 tx=0).stop 2024-04-23T18:35:28.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.315+0000 7f48f59d8640 1 -- 172.21.15.132:0/1685864986 shutdown_connections 2024-04-23T18:35:28.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.315+0000 7f48f59d8640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f48d80a3bb0 0x7f48d80a6070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.315+0000 7f48f59d8640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48f01029a0 0x7f48f019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.315+0000 7f48f59d8640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f0101fe0 0x7f48f019ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.315+0000 7f48f59d8640 1 --2- 172.21.15.132:0/1685864986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48f0100e90 0x7f48f0069c70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.315+0000 7f48f59d8640 1 -- 172.21.15.132:0/1685864986 >> 172.21.15.132:0/1685864986 conn(0x7f48f00fc400 msgr2=0x7f48f0105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:28.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.315+0000 7f48f59d8640 1 -- 172.21.15.132:0/1685864986 shutdown_connections 2024-04-23T18:35:28.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.315+0000 7f48f59d8640 1 -- 172.21.15.132:0/1685864986 wait complete. 2024-04-23T18:35:28.317 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:28 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2241307076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:28.317 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:28 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2340215281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:35:28.317 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:28 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1279751848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:35:28.318 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:35:28.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.423+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/437178033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8ac101fe0 msgr2=0x7fa8ac102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.423+0000 7fa8b2a6b640 1 --2- 172.21.15.132:0/437178033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8ac101fe0 0x7fa8ac102460 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa898009900 tx=0x7fa89802a410 comp rx=0 tx=0).stop 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.423+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/437178033 shutdown_connections 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.423+0000 7fa8b2a6b640 1 --2- 172.21.15.132:0/437178033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac1029a0 0x7fa8ac1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.423+0000 7fa8b2a6b640 1 --2- 172.21.15.132:0/437178033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8ac101fe0 0x7fa8ac102460 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.423+0000 7fa8b2a6b640 1 --2- 172.21.15.132:0/437178033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8ac100e90 0x7fa8ac1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.423+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/437178033 >> 172.21.15.132:0/437178033 conn(0x7fa8ac0fc400 msgr2=0x7fa8ac0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.423+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/437178033 shutdown_connections 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/437178033 wait complete. 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b2a6b640 1 Processor -- start 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b2a6b640 1 -- start start 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b2a6b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac100e90 0x7fa8ac1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b2a6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8ac101fe0 0x7fa8ac198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b2a6b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8ac1029a0 0x7fa8ac19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:28.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b2a6b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac10db30 con 0x7fa8ac1029a0 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b2a6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac10d9b0 con 0x7fa8ac101fe0 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b2a6b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac10dcb0 con 0x7fa8ac100e90 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b1008640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8ac1029a0 0x7fa8ac19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b1008640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8ac1029a0 0x7fa8ac19f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49288/0 (socket says 172.21.15.132:49288) 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b1008640 1 -- 172.21.15.132:0/2608634085 learned_addr learned my addr 172.21.15.132:0/2608634085 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8abfff640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8ac101fe0 0x7fa8ac198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.424+0000 7fa8b0807640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac100e90 0x7fa8ac1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.425+0000 7fa8b1008640 1 -- 172.21.15.132:0/2608634085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac100e90 msgr2=0x7fa8ac1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.425+0000 7fa8b1008640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac100e90 0x7fa8ac1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.425+0000 7fa8b1008640 1 -- 172.21.15.132:0/2608634085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8ac101fe0 msgr2=0x7fa8ac198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.425+0000 7fa8b1008640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8ac101fe0 0x7fa8ac198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.425+0000 7fa8b1008640 1 -- 172.21.15.132:0/2608634085 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ac19fe10 con 0x7fa8ac1029a0 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.425+0000 7fa8b1008640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8ac1029a0 0x7fa8ac19f7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400eaa0 tx=0x7fa8a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:28.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.425+0000 7fa8a9ffb640 1 -- 172.21.15.132:0/2608634085 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a400cdb0 con 0x7fa8ac1029a0 2024-04-23T18:35:28.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.425+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/2608634085 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ac1a0100 con 0x7fa8ac1029a0 2024-04-23T18:35:28.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.425+0000 7fa8a9ffb640 1 -- 172.21.15.132:0/2608634085 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a4005020 con 0x7fa8ac1029a0 2024-04-23T18:35:28.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.425+0000 7fa8a9ffb640 1 -- 172.21.15.132:0/2608634085 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a40116d0 con 0x7fa8ac1029a0 2024-04-23T18:35:28.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.425+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/2608634085 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ac1a0620 con 0x7fa8ac1029a0 2024-04-23T18:35:28.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.426+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/2608634085 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8ac1081f0 con 0x7fa8ac1029a0 2024-04-23T18:35:28.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.427+0000 7fa8a9ffb640 1 -- 172.21.15.132:0/2608634085 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fa8a40054a0 con 0x7fa8ac1029a0 2024-04-23T18:35:28.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.427+0000 7fa8a9ffb640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa89c0a3bb0 0x7fa89c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:28.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.428+0000 7fa8b0807640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa89c0a3bb0 0x7fa89c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:28.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.428+0000 7fa8a9ffb640 1 -- 172.21.15.132:0/2608634085 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa8a40d0640 con 0x7fa8ac1029a0 2024-04-23T18:35:28.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.428+0000 7fa8b0807640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa89c0a3bb0 0x7fa89c0a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00058c0 tx=0x7fa8a000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:28.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.432+0000 7fa8a9ffb640 1 -- 172.21.15.132:0/2608634085 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8a4016030 con 0x7fa8ac1029a0 2024-04-23T18:35:28.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:28 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2241307076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:28.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:28 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2340215281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:35:28.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:28 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1279751848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:35:28.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:28 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2241307076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:28.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:28 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2340215281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:35:28.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:28 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1279751848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:35:28.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.651+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/2608634085 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8ac062b50 con 0x7fa8ac1029a0 2024-04-23T18:35:28.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.651+0000 7fa8a9ffb640 1 -- 172.21.15.132:0/2608634085 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11946 (secure 0 0 0) 0x7fa8a409d010 con 0x7fa8ac1029a0 2024-04-23T18:35:28.643 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:28.644 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:34:07.536766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24394,24425,24460]},"id":1}]} 2024-04-23T18:35:28.644 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 25 2024-04-23T18:35:28.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.653+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/2608634085 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa89c0a3bb0 msgr2=0x7fa89c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.654+0000 7fa8b2a6b640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa89c0a3bb0 0x7fa89c0a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00058c0 tx=0x7fa8a000a4c0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.654+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/2608634085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8ac1029a0 msgr2=0x7fa8ac19f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.654+0000 7fa8b2a6b640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8ac1029a0 0x7fa8ac19f7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400eaa0 tx=0x7fa8a400ef60 comp rx=0 tx=0).stop 2024-04-23T18:35:28.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.654+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/2608634085 shutdown_connections 2024-04-23T18:35:28.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.654+0000 7fa8b2a6b640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa89c0a3bb0 0x7fa89c0a6070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.654+0000 7fa8b2a6b640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8ac1029a0 0x7fa8ac19f7a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.654+0000 7fa8b2a6b640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8ac101fe0 0x7fa8ac198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.654+0000 7fa8b2a6b640 1 --2- 172.21.15.132:0/2608634085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac100e90 0x7fa8ac1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.654+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/2608634085 >> 172.21.15.132:0/2608634085 conn(0x7fa8ac0fc400 msgr2=0x7fa8ac1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:28.647 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.654+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/2608634085 shutdown_connections 2024-04-23T18:35:28.647 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.654+0000 7fa8b2a6b640 1 -- 172.21.15.132:0/2608634085 wait complete. 2024-04-23T18:35:28.657 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:35:28.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2768248334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0acc1029b0 msgr2=0x7f0acc1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 --2- 172.21.15.132:0/2768248334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0acc1029b0 0x7f0acc1092d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009ca0 tx=0x7f0ac402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2768248334 shutdown_connections 2024-04-23T18:35:28.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 --2- 172.21.15.132:0/2768248334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0acc101ff0 0x7f0acc102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 --2- 172.21.15.132:0/2768248334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0acc1029b0 0x7f0acc1092d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 --2- 172.21.15.132:0/2768248334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0acc100ea0 0x7f0acc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2768248334 >> 172.21.15.132:0/2768248334 conn(0x7f0acc0fc450 msgr2=0x7f0acc0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2768248334 shutdown_connections 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2768248334 wait complete. 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 Processor -- start 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 -- start start 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0acc100ea0 0x7f0acc197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0acc101ff0 0x7f0acc1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0acc1029b0 0x7f0acc19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc10c9f0 con 0x7f0acc101ff0 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc10c870 con 0x7f0acc1029b0 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.763+0000 7f0ad1b6f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc10cb70 con 0x7f0acc100ea0 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0acc101ff0 0x7f0acc1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0acc101ff0 0x7f0acc1975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49302/0 (socket says 172.21.15.132:49302) 2024-04-23T18:35:28.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ac3fff640 1 -- 172.21.15.132:0/2932026810 learned_addr learned my addr 172.21.15.132:0/2932026810 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0acb7fe640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0acc100ea0 0x7f0acc197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0acbfff640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0acc1029b0 0x7f0acc19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ac3fff640 1 -- 172.21.15.132:0/2932026810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0acc100ea0 msgr2=0x7f0acc197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ac3fff640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0acc100ea0 0x7f0acc197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ac3fff640 1 -- 172.21.15.132:0/2932026810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0acc1029b0 msgr2=0x7f0acc19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ac3fff640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0acc1029b0 0x7f0acc19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ac3fff640 1 -- 172.21.15.132:0/2932026810 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0acc19ec80 con 0x7f0acc101ff0 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ac3fff640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0acc101ff0 0x7f0acc1975a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400d950 tx=0x7f0ab400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ac97fa640 1 -- 172.21.15.132:0/2932026810 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4017070 con 0x7f0acc101ff0 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ac97fa640 1 -- 172.21.15.132:0/2932026810 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab4009c30 con 0x7f0acc101ff0 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2932026810 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0acc19ef70 con 0x7f0acc101ff0 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ac97fa640 1 -- 172.21.15.132:0/2932026810 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400f620 con 0x7f0acc101ff0 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.764+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2932026810 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0acc19f3e0 con 0x7f0acc101ff0 2024-04-23T18:35:28.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.765+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2932026810 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0acc1081b0 con 0x7f0acc101ff0 2024-04-23T18:35:28.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.766+0000 7f0ac97fa640 1 -- 172.21.15.132:0/2932026810 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f0ab401a070 con 0x7f0acc101ff0 2024-04-23T18:35:28.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.767+0000 7f0ac97fa640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f0ab80a3bb0 0x7f0ab80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:28.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.767+0000 7f0acb7fe640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f0ab80a3bb0 0x7f0ab80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:28.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.767+0000 7f0ac97fa640 1 -- 172.21.15.132:0/2932026810 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0ab40cfde0 con 0x7f0acc101ff0 2024-04-23T18:35:28.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.767+0000 7f0acb7fe640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f0ab80a3bb0 0x7f0ab80a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00bbe0 tx=0x7f0abc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:28.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.771+0000 7f0ac97fa640 1 -- 172.21.15.132:0/2932026810 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ab4009de0 con 0x7f0acc101ff0 2024-04-23T18:35:28.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.948+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2932026810 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0acc02d810 con 0x7f0acc101ff0 2024-04-23T18:35:29.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.949+0000 7f0ac97fa640 1 -- 172.21.15.132:0/2932026810 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22800 (secure 0 0 0) 0x7f0ab409c7b0 con 0x7f0acc101ff0 2024-04-23T18:35:29.019 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:29.020 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":88,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:34:55.248852+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:35:29.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.951+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2932026810 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f0ab80a3bb0 msgr2=0x7f0ab80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.951+0000 7f0ad1b6f640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f0ab80a3bb0 0x7f0ab80a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00bbe0 tx=0x7f0abc00b390 comp rx=0 tx=0).stop 2024-04-23T18:35:29.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.951+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2932026810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0acc101ff0 msgr2=0x7f0acc1975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.951+0000 7f0ad1b6f640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0acc101ff0 0x7f0acc1975a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400d950 tx=0x7f0ab400de10 comp rx=0 tx=0).stop 2024-04-23T18:35:29.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.952+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2932026810 shutdown_connections 2024-04-23T18:35:29.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.952+0000 7f0ad1b6f640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f0ab80a3bb0 0x7f0ab80a6070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.952+0000 7f0ad1b6f640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0acc1029b0 0x7f0acc19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.952+0000 7f0ad1b6f640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0acc101ff0 0x7f0acc1975a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.952+0000 7f0ad1b6f640 1 --2- 172.21.15.132:0/2932026810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0acc100ea0 0x7f0acc197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.952+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2932026810 >> 172.21.15.132:0/2932026810 conn(0x7f0acc0fc450 msgr2=0x7f0acc105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.952+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2932026810 shutdown_connections 2024-04-23T18:35:29.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:28.952+0000 7f0ad1b6f640 1 -- 172.21.15.132:0/2932026810 wait complete. 2024-04-23T18:35:29.025 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:35:29.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.131+0000 7f0f95bcd640 1 -- 172.21.15.132:0/984675353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f901032e0 msgr2=0x7f0f90103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.131+0000 7f0f95bcd640 1 --2- 172.21.15.132:0/984675353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f901032e0 0x7f0f90103760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009900 tx=0x7f0f8402a400 comp rx=0 tx=0).stop 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.131+0000 7f0f95bcd640 1 -- 172.21.15.132:0/984675353 shutdown_connections 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.131+0000 7f0f95bcd640 1 --2- 172.21.15.132:0/984675353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f90103ca0 0x7f0f9010a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.131+0000 7f0f95bcd640 1 --2- 172.21.15.132:0/984675353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f901032e0 0x7f0f90103760 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.131+0000 7f0f95bcd640 1 --2- 172.21.15.132:0/984675353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f90100270 0x7f0f90100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.131+0000 7f0f95bcd640 1 -- 172.21.15.132:0/984675353 >> 172.21.15.132:0/984675353 conn(0x7f0f900fbbc0 msgr2=0x7f0f900fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.131+0000 7f0f95bcd640 1 -- 172.21.15.132:0/984675353 shutdown_connections 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.131+0000 7f0f95bcd640 1 -- 172.21.15.132:0/984675353 wait complete. 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.131+0000 7f0f95bcd640 1 Processor -- start 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f95bcd640 1 -- start start 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f95bcd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f90100270 0x7f0f901981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f95bcd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f901032e0 0x7f0f90198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f95bcd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f90103ca0 0x7f0f9019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f95bcd640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9010db10 con 0x7f0f90103ca0 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f95bcd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9010d990 con 0x7f0f90100270 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f95bcd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9010dc90 con 0x7f0f901032e0 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f90103ca0 0x7f0f9019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f90103ca0 0x7f0f9019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49322/0 (socket says 172.21.15.132:49322) 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8ffff640 1 -- 172.21.15.132:0/3972549329 learned_addr learned my addr 172.21.15.132:0/3972549329 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8f7fe640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f90100270 0x7f0f901981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8ffff640 1 -- 172.21.15.132:0/3972549329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f901032e0 msgr2=0x7f0f90198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8effd640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f901032e0 0x7f0f90198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8ffff640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f901032e0 0x7f0f90198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8ffff640 1 -- 172.21.15.132:0/3972549329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f90100270 msgr2=0x7f0f901981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8ffff640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f90100270 0x7f0f901981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8ffff640 1 -- 172.21.15.132:0/3972549329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9019fdf0 con 0x7f0f90103ca0 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8f7fe640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f90100270 0x7f0f901981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8ffff640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f90103ca0 0x7f0f9019f780 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800eaa0 tx=0x7f0f8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.132+0000 7f0f8cff9640 1 -- 172.21.15.132:0/3972549329 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8800cdb0 con 0x7f0f90103ca0 2024-04-23T18:35:29.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.133+0000 7f0f95bcd640 1 -- 172.21.15.132:0/3972549329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f901a00e0 con 0x7f0f90103ca0 2024-04-23T18:35:29.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.133+0000 7f0f8cff9640 1 -- 172.21.15.132:0/3972549329 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f88005020 con 0x7f0f90103ca0 2024-04-23T18:35:29.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.133+0000 7f0f8cff9640 1 -- 172.21.15.132:0/3972549329 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f880116d0 con 0x7f0f90103ca0 2024-04-23T18:35:29.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.133+0000 7f0f95bcd640 1 -- 172.21.15.132:0/3972549329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f901a0630 con 0x7f0f90103ca0 2024-04-23T18:35:29.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.133+0000 7f0f95bcd640 1 -- 172.21.15.132:0/3972549329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f901094e0 con 0x7f0f90103ca0 2024-04-23T18:35:29.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.135+0000 7f0f8cff9640 1 -- 172.21.15.132:0/3972549329 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f0f880051d0 con 0x7f0f90103ca0 2024-04-23T18:35:29.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.135+0000 7f0f8cff9640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f0f780a3bb0 0x7f0f780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.135+0000 7f0f8f7fe640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f0f780a3bb0 0x7f0f780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.135+0000 7f0f8cff9640 1 -- 172.21.15.132:0/3972549329 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0f880d0a10 con 0x7f0f90103ca0 2024-04-23T18:35:29.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.136+0000 7f0f8f7fe640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f0f780a3bb0 0x7f0f780a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0f80005d00 tx=0x7f0f80005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.139+0000 7f0f8cff9640 1 -- 172.21.15.132:0/3972549329 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f88016030 con 0x7f0f90103ca0 2024-04-23T18:35:29.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.358+0000 7f0f95bcd640 1 -- 172.21.15.132:0/3972549329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f9002d5c0 con 0x7f0f90103ca0 2024-04-23T18:35:29.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.359+0000 7f0f8cff9640 1 -- 172.21.15.132:0/3972549329 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11946 (secure 0 0 0) 0x7f0f8809d3e0 con 0x7f0f90103ca0 2024-04-23T18:35:29.351 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:29.351 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:34:07.536766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24394,24425,24460]},"id":1}]} 2024-04-23T18:35:29.351 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 25 2024-04-23T18:35:29.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.361+0000 7f0f95bcd640 1 -- 172.21.15.132:0/3972549329 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f0f780a3bb0 msgr2=0x7f0f780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.361+0000 7f0f95bcd640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f0f780a3bb0 0x7f0f780a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0f80005d00 tx=0x7f0f80005c90 comp rx=0 tx=0).stop 2024-04-23T18:35:29.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.361+0000 7f0f95bcd640 1 -- 172.21.15.132:0/3972549329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f90103ca0 msgr2=0x7f0f9019f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.361+0000 7f0f95bcd640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f90103ca0 0x7f0f9019f780 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800eaa0 tx=0x7f0f8800ef60 comp rx=0 tx=0).stop 2024-04-23T18:35:29.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.361+0000 7f0f95bcd640 1 -- 172.21.15.132:0/3972549329 shutdown_connections 2024-04-23T18:35:29.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.361+0000 7f0f95bcd640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f0f780a3bb0 0x7f0f780a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.361+0000 7f0f95bcd640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f901032e0 0x7f0f90198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.361+0000 7f0f95bcd640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f90103ca0 0x7f0f9019f780 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.361+0000 7f0f95bcd640 1 --2- 172.21.15.132:0/3972549329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f90100270 0x7f0f901981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.361+0000 7f0f95bcd640 1 -- 172.21.15.132:0/3972549329 >> 172.21.15.132:0/3972549329 conn(0x7f0f900fbbc0 msgr2=0x7f0f901066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.361+0000 7f0f95bcd640 1 -- 172.21.15.132:0/3972549329 shutdown_connections 2024-04-23T18:35:29.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.361+0000 7f0f95bcd640 1 -- 172.21.15.132:0/3972549329 wait complete. 2024-04-23T18:35:29.365 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T18:34:00.571339+0000', 'modified': '2024-04-23T18:34:07.536766+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24515, 'mds_1': 24491, 'mds_2': 24460, 'mds_3': 24425, 'mds_4': 24394}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6839/579251026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 579251026}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 579251026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6841/826931131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6840', 'nonce': 826931131}, {'type': 'v1', 'addr': '172.21.15.132:6841', 'nonce': 826931131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24394': {'gid': 24394, 'name': 'l', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.157:6833/1034092547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 1034092547}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 1034092547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.148:6833/3556073608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 3556073608}, {'type': 'v1', 'addr': '172.21.15.148:6833', 'nonce': 3556073608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/4121037781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 4121037781}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 4121037781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6837/2670322328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 2670322328}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 2670322328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/397710093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 397710093}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 397710093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.157:6839/1528717147', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 1528717147}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 1528717147}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.148:6837/2626530887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 2626530887}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 2626530887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6839/4115412726', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 4115412726}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 4115412726}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24515, 'qdb_cluster': [24491, 24515, 24394, 24425, 24460]} 2024-04-23T18:35:29.365 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-23T18:35:29.365 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-23T18:35:29.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.511+0000 7f527e7af640 1 -- 172.21.15.132:0/2812918061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278102440 msgr2=0x7f5278102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.511+0000 7f527e7af640 1 --2- 172.21.15.132:0/2812918061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278102440 0x7f5278102860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5268009900 tx=0x7f526802a410 comp rx=0 tx=0).stop 2024-04-23T18:35:29.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.511+0000 7f527e7af640 1 -- 172.21.15.132:0/2812918061 shutdown_connections 2024-04-23T18:35:29.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.511+0000 7f527e7af640 1 --2- 172.21.15.132:0/2812918061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278101170 0x7f52781015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.511+0000 7f527e7af640 1 --2- 172.21.15.132:0/2812918061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278101b30 0x7f52781093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.511+0000 7f527e7af640 1 --2- 172.21.15.132:0/2812918061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278102440 0x7f5278102860 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.511+0000 7f527e7af640 1 -- 172.21.15.132:0/2812918061 >> 172.21.15.132:0/2812918061 conn(0x7f52780fbbc0 msgr2=0x7f52780fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.511+0000 7f527e7af640 1 -- 172.21.15.132:0/2812918061 shutdown_connections 2024-04-23T18:35:29.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.511+0000 7f527e7af640 1 -- 172.21.15.132:0/2812918061 wait complete. 2024-04-23T18:35:29.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.512+0000 7f527e7af640 1 Processor -- start 2024-04-23T18:35:29.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.512+0000 7f527e7af640 1 -- start start 2024-04-23T18:35:29.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.512+0000 7f527e7af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278101170 0x7f52781982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.512+0000 7f527e7af640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278101b30 0x7f52781987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.512+0000 7f527e7af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278102440 0x7f527819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.512+0000 7f527e7af640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f527810db70 con 0x7f5278101b30 2024-04-23T18:35:29.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.512+0000 7f527e7af640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f527810d9f0 con 0x7f5278102440 2024-04-23T18:35:29.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.512+0000 7f527e7af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f527810dcf0 con 0x7f5278101170 2024-04-23T18:35:29.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.512+0000 7f52777fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278101b30 0x7f52781987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.512+0000 7f52777fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278101b30 0x7f52781987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34534/0 (socket says 172.21.15.132:34534) 2024-04-23T18:35:29.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.512+0000 7f52777fe640 1 -- 172.21.15.132:0/3174999998 learned_addr learned my addr 172.21.15.132:0/3174999998 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f5277fff640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278101170 0x7f52781982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f52777fe640 1 -- 172.21.15.132:0/3174999998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278101170 msgr2=0x7f52781982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f52777fe640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278101170 0x7f52781982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f52777fe640 1 -- 172.21.15.132:0/3174999998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278102440 msgr2=0x7f527819f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f527cd4c640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278102440 0x7f527819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f52777fe640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278102440 0x7f527819f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f52777fe640 1 -- 172.21.15.132:0/3174999998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527819fec0 con 0x7f5278101b30 2024-04-23T18:35:29.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f527cd4c640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278102440 0x7f527819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:29.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f52777fe640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278101b30 0x7f52781987e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f526c00d770 tx=0x7f526c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f52757fa640 1 -- 172.21.15.132:0/3174999998 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c017070 con 0x7f5278101b30 2024-04-23T18:35:29.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f52757fa640 1 -- 172.21.15.132:0/3174999998 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526c0054d0 con 0x7f5278101b30 2024-04-23T18:35:29.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f527e7af640 1 -- 172.21.15.132:0/3174999998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52781a01b0 con 0x7f5278101b30 2024-04-23T18:35:29.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f52757fa640 1 -- 172.21.15.132:0/3174999998 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c004c30 con 0x7f5278101b30 2024-04-23T18:35:29.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.513+0000 7f527e7af640 1 -- 172.21.15.132:0/3174999998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52781a0730 con 0x7f5278101b30 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f52757fa640 1 -- 172.21.15.132:0/3174999998 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f526c004e10 con 0x7f5278101b30 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f528159c740 1 Processor -- start 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f528159c740 1 -- start start 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f528159c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd3b80 0x55d841bd3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f528159c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd4d50 0x55d841bd51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f528159c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd5710 0x55d841bdbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f528159c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55d841b38c50 con 0x55d841bd5710 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f528159c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d841ad5830 con 0x55d841bd3b80 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f528159c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d841b34c10 con 0x55d841bd4d50 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f52777fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd5710 0x55d841bdbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f52777fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd5710 0x55d841bdbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34550/0 (socket says 172.21.15.132:34550) 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f52777fe640 1 -- 172.21.15.132:0/1867084074 learned_addr learned my addr 172.21.15.132:0/1867084074 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f5277fff640 1 --2- 172.21.15.132:0/1867084074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd4d50 0x55d841bd51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f52757fa640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52600a3bb0 0x7f52600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f527cd4c640 1 --2- 172.21.15.132:0/1867084074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd3b80 0x55d841bd3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f52777fe640 1 -- 172.21.15.132:0/1867084074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd4d50 msgr2=0x55d841bd51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f52777fe640 1 --2- 172.21.15.132:0/1867084074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd4d50 0x55d841bd51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f52777fe640 1 -- 172.21.15.132:0/1867084074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd3b80 msgr2=0x55d841bd3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f52777fe640 1 --2- 172.21.15.132:0/1867084074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd3b80 0x55d841bd3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f52777fe640 1 -- 172.21.15.132:0/1867084074 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d84184b2d0 con 0x55d841bd5710 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.516+0000 7f527cd4c640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52600a3bb0 0x7f52600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.517+0000 7f52757fa640 1 -- 172.21.15.132:0/3174999998 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f526c0cff80 con 0x7f5278101b30 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.517+0000 7f52777fe640 1 --2- 172.21.15.132:0/1867084074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd5710 0x55d841bdbfa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f526c046f30 tx=0x7f526c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc90210ec299dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.517+0000 7f525e7fc640 1 -- 172.21.15.132:0/1867084074 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c09bd90 con 0x55d841bd5710 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.517+0000 7f525e7fc640 1 -- 172.21.15.132:0/1867084074 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526c005680 con 0x55d841bd5710 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.517+0000 7f527cd4c640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52600a3bb0 0x7f52600a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55d841bd4c50 tx=0x7f5270004b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.517+0000 7f525e7fc640 1 -- 172.21.15.132:0/1867084074 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c0d3770 con 0x55d841bd5710 2024-04-23T18:35:29.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.517+0000 7f528159c740 1 -- 172.21.15.132:0/1867084074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd5710 msgr2=0x55d841bdbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.517+0000 7f528159c740 1 --2- 172.21.15.132:0/1867084074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd5710 0x55d841bdbfa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f526c046f30 tx=0x7f526c022000 comp rx=0 tx=0).stop 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 -- 172.21.15.132:0/1867084074 shutdown_connections 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 --2- 172.21.15.132:0/1867084074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd4d50 0x55d841bd51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 --2- 172.21.15.132:0/1867084074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd5710 0x55d841bdbfa0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 --2- 172.21.15.132:0/1867084074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd3b80 0x55d841bd3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 -- 172.21.15.132:0/1867084074 >> 172.21.15.132:0/1867084074 conn(0x55d841bcf070 msgr2=0x55d841bcf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 -- 172.21.15.132:0/1867084074 shutdown_connections 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 -- 172.21.15.132:0/1867084074 wait complete. 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 -- start start 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 Processor -- start 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd3b80 0x55d841bddf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd4d50 0x55d841bde490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd5710 0x55d841be54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55d841bdc6c0 con 0x55d841bd4d50 2024-04-23T18:35:29.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d841ad3470 con 0x55d841bd5710 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d841bdc840 con 0x55d841bd3b80 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f528159c740 10 client.0 _ll_register_callbacks cb 0x55d841af8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f52777fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd4d50 0x55d841bde490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f52777fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd4d50 0x55d841bde490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34560/0 (socket says 172.21.15.132:34560) 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.518+0000 7f52777fe640 1 -- 172.21.15.132:0/3570184703 learned_addr learned my addr 172.21.15.132:0/3570184703 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f5277fff640 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd3b80 0x55d841bddf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f527cd4c640 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd5710 0x55d841be54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f52777fe640 1 -- 172.21.15.132:0/3570184703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd3b80 msgr2=0x55d841bddf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f52777fe640 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd3b80 0x55d841bddf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f52777fe640 1 -- 172.21.15.132:0/3570184703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd5710 msgr2=0x55d841be54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f52777fe640 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd5710 0x55d841be54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f52777fe640 1 -- 172.21.15.132:0/3570184703 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8419156b0 con 0x55d841bd4d50 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f52777fe640 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd4d50 0x55d841bde490 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f526c0876e0 tx=0x7f526c09be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2680b8a300f17d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f52577fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f52577fe640 1 -- 172.21.15.132:0/3570184703 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c016170 con 0x55d841bd4d50 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f52577fe640 1 -- 172.21.15.132:0/3570184703 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f526c09a2f0 con 0x55d841bd4d50 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f528159c740 1 -- 172.21.15.132:0/3570184703 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f526c005680 con 0x55d841bd4d50 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f52577fe640 1 -- 172.21.15.132:0/3570184703 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f526c09bbb0 con 0x55d841bd4d50 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f528159c740 10 client.15066 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f528159c740 1 -- 172.21.15.132:0/3570184703 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d841be8840 con 0x55d841bd4d50 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f52577fe640 1 -- 172.21.15.132:0/3570184703 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526c005680 con 0x55d841bd4d50 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f52577fe640 1 -- 172.21.15.132:0/3570184703 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f526c0d3690 con 0x55d841bd4d50 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f528159c740 10 client.15066 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f528159c740 10 client.15066 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f528159c740 20 client.15066 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f528159c740 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x55d841beaa50 0x55d841bebec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f528159c740 4 client.15066 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:29.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.519+0000 7f528159c740 1 -- 172.21.15.132:0/3570184703 --> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d841b46670 con 0x55d841beaa50 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.520+0000 7f5277fff640 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x55d841beaa50 0x55d841bebec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.520+0000 7f5277fff640 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x55d841beaa50 0x55d841bebec0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ba3214b8f31774a8 server_cookie=ca1d77f880ec3724 in_seq=0 out_seq=0 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.520+0000 7f52577fe640 10 client.15066 ms_handle_connect on v2:172.21.15.157:6832/1034092547 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f52577fe640 1 -- 172.21.15.132:0/3570184703 <== mds.4 v2:172.21.15.157:6832/1034092547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d841b46670 con 0x55d841beaa50 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f52577fe640 10 client.15066 handle_command_reply: tid=0 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f528159c740 1 client.15066 shutdown 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f528159c740 1 -- 172.21.15.132:0/3570184703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd4d50 msgr2=0x55d841bde490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f528159c740 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd4d50 0x55d841bde490 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f526c0876e0 tx=0x7f526c09be80 comp rx=0 tx=0).stop 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f528159c740 1 -- 172.21.15.132:0/3570184703 shutdown_connections 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f528159c740 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x55d841beaa50 0x55d841bebec0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f528159c740 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd5710 0x55d841be54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f528159c740 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd4d50 0x55d841bde490 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f528159c740 1 --2- 172.21.15.132:0/3570184703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd3b80 0x55d841bddf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f52577fe640 0 client.15066 ms_handle_reset on v2:172.21.15.157:6832/1034092547 2024-04-23T18:35:29.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f528159c740 1 -- 172.21.15.132:0/3570184703 >> 172.21.15.132:0/3570184703 conn(0x55d841bcf070 msgr2=0x55d841bce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f528159c740 1 -- 172.21.15.132:0/3570184703 shutdown_connections 2024-04-23T18:35:29.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f528159c740 1 -- 172.21.15.132:0/3570184703 wait complete. 2024-04-23T18:35:29.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.521+0000 7f528159c740 20 client.15066 trim_cache size 0 max 16384 2024-04-23T18:35:29.521 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:29 smithi148 ceph-mon[32311]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T18:35:29.521 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:29 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1685864986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:35:29.521 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:29 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2608634085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:29.521 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:29 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2932026810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:29.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f528159c740 1 Processor -- start 2024-04-23T18:35:29.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f528159c740 1 -- start start 2024-04-23T18:35:29.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f528159c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 0x55d841bce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f528159c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd4d50 0x55d841bce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f528159c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd5710 0x55d841be5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f528159c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55d841bea200 con 0x55d841bd3b80 2024-04-23T18:35:29.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f528159c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d841bea8d0 con 0x55d841bd5710 2024-04-23T18:35:29.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f528159c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d841beafa0 con 0x55d841bd4d50 2024-04-23T18:35:29.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f5277fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 0x55d841bce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f5277fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 0x55d841bce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34570/0 (socket says 172.21.15.132:34570) 2024-04-23T18:35:29.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f5277fff640 1 -- 172.21.15.132:0/2161137268 learned_addr learned my addr 172.21.15.132:0/2161137268 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f5277fff640 1 -- 172.21.15.132:0/2161137268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd4d50 msgr2=0x55d841bce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f5277fff640 1 --2- 172.21.15.132:0/2161137268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd4d50 0x55d841bce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f5277fff640 1 -- 172.21.15.132:0/2161137268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd5710 msgr2=0x55d841be5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f5277fff640 1 --2- 172.21.15.132:0/2161137268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd5710 0x55d841be5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f5277fff640 1 -- 172.21.15.132:0/2161137268 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d841bd47d0 con 0x55d841bd3b80 2024-04-23T18:35:29.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f5277fff640 1 --2- 172.21.15.132:0/2161137268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 0x55d841bce050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5268000c00 tx=0x7f52680314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfc12c3487b3d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f525dffb640 1 -- 172.21.15.132:0/2161137268 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268048070 con 0x55d841bd3b80 2024-04-23T18:35:29.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f525dffb640 1 -- 172.21.15.132:0/2161137268 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5268004310 con 0x55d841bd3b80 2024-04-23T18:35:29.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.540+0000 7f525dffb640 1 -- 172.21.15.132:0/2161137268 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268035040 con 0x55d841bd3b80 2024-04-23T18:35:29.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.541+0000 7f528159c740 1 -- 172.21.15.132:0/2161137268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 msgr2=0x55d841bce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.541+0000 7f528159c740 1 --2- 172.21.15.132:0/2161137268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 0x55d841bce050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5268000c00 tx=0x7f52680314a0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.541+0000 7f528159c740 1 -- 172.21.15.132:0/2161137268 shutdown_connections 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.541+0000 7f528159c740 1 --2- 172.21.15.132:0/2161137268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd4d50 0x55d841bce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.541+0000 7f528159c740 1 --2- 172.21.15.132:0/2161137268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd5710 0x55d841be5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.541+0000 7f528159c740 1 --2- 172.21.15.132:0/2161137268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 0x55d841bce050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.541+0000 7f528159c740 1 -- 172.21.15.132:0/2161137268 >> 172.21.15.132:0/2161137268 conn(0x55d841bcf070 msgr2=0x55d841bcd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.541+0000 7f528159c740 1 -- 172.21.15.132:0/2161137268 shutdown_connections 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.541+0000 7f528159c740 1 -- 172.21.15.132:0/2161137268 wait complete. 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.541+0000 7f528159c740 1 -- start start 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.541+0000 7f528159c740 1 Processor -- start 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f528159c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 0x55d841be1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f528159c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd4d50 0x55d841be1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f528159c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd5710 0x55d841be47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f528159c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55d841bebd40 con 0x55d841bd3b80 2024-04-23T18:35:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f528159c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d841beb670 con 0x55d841bd4d50 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f528159c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d841bec410 con 0x55d841bd5710 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f528159c740 10 client.0 _ll_register_callbacks cb 0x7f5268009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f5277fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 0x55d841be1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f5277fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 0x55d841be1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34584/0 (socket says 172.21.15.132:34584) 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f5277fff640 1 -- 172.21.15.132:0/2445512837 learned_addr learned my addr 172.21.15.132:0/2445512837 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f5277fff640 1 -- 172.21.15.132:0/2445512837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd5710 msgr2=0x55d841be47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f5277fff640 1 --2- 172.21.15.132:0/2445512837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd5710 0x55d841be47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f5277fff640 1 -- 172.21.15.132:0/2445512837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd4d50 msgr2=0x55d841be1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f5277fff640 1 --2- 172.21.15.132:0/2445512837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd4d50 0x55d841be1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f5277fff640 1 -- 172.21.15.132:0/2445512837 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d841be4dc0 con 0x55d841bd3b80 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f5277fff640 1 --2- 172.21.15.132:0/2445512837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 0x55d841be1350 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5268005ab0 tx=0x7f5268004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bfb9b9257614e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f52577fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f52577fe640 1 -- 172.21.15.132:0/2445512837 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268035070 con 0x55d841bd3b80 2024-04-23T18:35:29.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f52577fe640 1 -- 172.21.15.132:0/2445512837 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5268031840 con 0x55d841bd3b80 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f528159c740 1 -- 172.21.15.132:0/2445512837 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5268004310 con 0x55d841bd3b80 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f52577fe640 1 -- 172.21.15.132:0/2445512837 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5268048040 con 0x55d841bd3b80 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f528159c740 10 client.15072 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.542+0000 7f528159c740 1 -- 172.21.15.132:0/2445512837 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d841c51000 con 0x55d841bd3b80 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.543+0000 7f52577fe640 1 -- 172.21.15.132:0/2445512837 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5268004310 con 0x55d841bd3b80 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.543+0000 7f52577fe640 1 -- 172.21.15.132:0/2445512837 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5268055020 con 0x55d841bd3b80 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.543+0000 7f528159c740 10 client.15072 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.543+0000 7f528159c740 10 client.15072 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.543+0000 7f528159c740 20 client.15072 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.543+0000 7f528159c740 1 --2- 172.21.15.132:0/2445512837 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x55d841c55790 0x55d841c544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.543+0000 7f528159c740 4 client.15072 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.543+0000 7f528159c740 1 -- 172.21.15.132:0/2445512837 --> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] -- command(tid 0: {"prefix": "status"}) -- 0x55d841b404e0 con 0x55d841c55790 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.543+0000 7f52777fe640 1 --2- 172.21.15.132:0/2445512837 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x55d841c55790 0x55d841c544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.543+0000 7f52777fe640 1 --2- 172.21.15.132:0/2445512837 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x55d841c55790 0x55d841c544a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=667933cd248c919d server_cookie=ecf0cdfeb7088c3e in_seq=0 out_seq=0 2024-04-23T18:35:29.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.543+0000 7f52577fe640 10 client.15072 ms_handle_connect on v2:172.21.15.157:6832/1034092547 2024-04-23T18:35:29.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f52577fe640 1 -- 172.21.15.132:0/2445512837 <== mds.4 v2:172.21.15.157:6832/1034092547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d841b404e0 con 0x55d841c55790 2024-04-23T18:35:29.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f52577fe640 10 client.15072 handle_command_reply: tid=0 2024-04-23T18:35:29.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f528159c740 1 client.15072 shutdown 2024-04-23T18:35:29.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f528159c740 1 -- 172.21.15.132:0/2445512837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 msgr2=0x55d841be1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f528159c740 1 --2- 172.21.15.132:0/2445512837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 0x55d841be1350 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5268005ab0 tx=0x7f5268004060 comp rx=0 tx=0).stop 2024-04-23T18:35:29.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f528159c740 1 -- 172.21.15.132:0/2445512837 shutdown_connections 2024-04-23T18:35:29.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f528159c740 1 --2- 172.21.15.132:0/2445512837 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x55d841c55790 0x55d841c544a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f528159c740 1 --2- 172.21.15.132:0/2445512837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d841bd5710 0x55d841be47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f52577fe640 0 client.15072 ms_handle_reset on v2:172.21.15.157:6832/1034092547 2024-04-23T18:35:29.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f528159c740 1 --2- 172.21.15.132:0/2445512837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d841bd4d50 0x55d841be1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f528159c740 1 --2- 172.21.15.132:0/2445512837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d841bd3b80 0x55d841be1350 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f528159c740 1 -- 172.21.15.132:0/2445512837 >> 172.21.15.132:0/2445512837 conn(0x55d841bcf070 msgr2=0x55d841bce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f528159c740 1 -- 172.21.15.132:0/2445512837 shutdown_connections 2024-04-23T18:35:29.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f528159c740 1 -- 172.21.15.132:0/2445512837 wait complete. 2024-04-23T18:35:29.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f528159c740 20 client.15072 trim_cache size 0 max 16384 2024-04-23T18:35:29.543 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:35:29.543 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:35:29.543 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 4, 2024-04-23T18:35:29.543 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24394, 2024-04-23T18:35:29.543 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:35:29.543 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:35:29.544 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:35:29.544 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 83.010390952999998, 2024-04-23T18:35:29.544 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 25, 2024-04-23T18:35:29.544 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 88, 2024-04-23T18:35:29.544 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:35:29.544 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 126.80515940399999 2024-04-23T18:35:29.544 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:35:29.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f527e7af640 1 -- 172.21.15.132:0/3174999998 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52600a3bb0 msgr2=0x7f52600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f527e7af640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52600a3bb0 0x7f52600a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55d841bd4c50 tx=0x7f5270004b50 comp rx=0 tx=0).stop 2024-04-23T18:35:29.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f527e7af640 1 -- 172.21.15.132:0/3174999998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278101b30 msgr2=0x7f52781987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.544+0000 7f527e7af640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278101b30 0x7f52781987e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f526c00d770 tx=0x7f526c00dc30 comp rx=0 tx=0).stop 2024-04-23T18:35:29.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.545+0000 7f527e7af640 1 -- 172.21.15.132:0/3174999998 shutdown_connections 2024-04-23T18:35:29.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.545+0000 7f527e7af640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52600a3bb0 0x7f52600a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.545+0000 7f527e7af640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5278102440 0x7f527819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.545+0000 7f527e7af640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278101b30 0x7f52781987e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.545+0000 7f527e7af640 1 --2- 172.21.15.132:0/3174999998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5278101170 0x7f52781982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.545+0000 7f527e7af640 1 -- 172.21.15.132:0/3174999998 >> 172.21.15.132:0/3174999998 conn(0x7f52780fbbc0 msgr2=0x7f52781055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.545+0000 7f527e7af640 1 -- 172.21.15.132:0/3174999998 shutdown_connections 2024-04-23T18:35:29.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.545+0000 7f527e7af640 1 -- 172.21.15.132:0/3174999998 wait complete. 2024-04-23T18:35:29.546 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:29 smithi157 ceph-mon[32580]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T18:35:29.546 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:29 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1685864986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:35:29.546 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:29 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2608634085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:29.546 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:29 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2932026810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:29.547 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:29 smithi132 ceph-mon[29178]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T18:35:29.547 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:29 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1685864986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:35:29.547 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:29 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2608634085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:29.547 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:29 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2932026810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:29.548 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-23T18:35:29.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.655+0000 7f6b72813640 1 -- 172.21.15.132:0/3155553453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b6c100e90 msgr2=0x7f6b6c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.655+0000 7f6b72813640 1 --2- 172.21.15.132:0/3155553453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b6c100e90 0x7f6b6c1012b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009900 tx=0x7f6b5c02a410 comp rx=0 tx=0).stop 2024-04-23T18:35:29.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.655+0000 7f6b72813640 1 -- 172.21.15.132:0/3155553453 shutdown_connections 2024-04-23T18:35:29.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.655+0000 7f6b72813640 1 --2- 172.21.15.132:0/3155553453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6c101fe0 0x7f6b6c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.655+0000 7f6b72813640 1 --2- 172.21.15.132:0/3155553453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b6c1029a0 0x7f6b6c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.655+0000 7f6b72813640 1 --2- 172.21.15.132:0/3155553453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b6c100e90 0x7f6b6c1012b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.655+0000 7f6b72813640 1 -- 172.21.15.132:0/3155553453 >> 172.21.15.132:0/3155553453 conn(0x7f6b6c0fc400 msgr2=0x7f6b6c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.655+0000 7f6b72813640 1 -- 172.21.15.132:0/3155553453 shutdown_connections 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.655+0000 7f6b72813640 1 -- 172.21.15.132:0/3155553453 wait complete. 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b72813640 1 Processor -- start 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b72813640 1 -- start start 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b72813640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b6c100e90 0x7f6b6c1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b72813640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b6c101fe0 0x7f6b6c197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b72813640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6c1029a0 0x7f6b6c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b72813640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10c9e0 con 0x7f6b6c100e90 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b72813640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10c860 con 0x7f6b6c101fe0 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b72813640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10cb60 con 0x7f6b6c1029a0 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b6bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b6c100e90 0x7f6b6c1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b6bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b6c100e90 0x7f6b6c1970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34606/0 (socket says 172.21.15.132:34606) 2024-04-23T18:35:29.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b6bfff640 1 -- 172.21.15.132:0/1843429579 learned_addr learned my addr 172.21.15.132:0/1843429579 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b6bfff640 1 -- 172.21.15.132:0/1843429579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6c1029a0 msgr2=0x7f6b6c19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6c1029a0 0x7f6b6c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b6bfff640 1 -- 172.21.15.132:0/1843429579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b6c101fe0 msgr2=0x7f6b6c197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b6b7fe640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b6c101fe0 0x7f6b6c197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b6c101fe0 0x7f6b6c197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b6bfff640 1 -- 172.21.15.132:0/1843429579 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c19ece0 con 0x7f6b6c100e90 2024-04-23T18:35:29.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.656+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b6c100e90 0x7f6b6c1970c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c005940 tx=0x7f6b5c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.657+0000 7f6b697fa640 1 -- 172.21.15.132:0/1843429579 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c035070 con 0x7f6b6c100e90 2024-04-23T18:35:29.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.657+0000 7f6b72813640 1 -- 172.21.15.132:0/1843429579 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6c06a5b0 con 0x7f6b6c100e90 2024-04-23T18:35:29.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.657+0000 7f6b697fa640 1 -- 172.21.15.132:0/1843429579 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b5c003ca0 con 0x7f6b6c100e90 2024-04-23T18:35:29.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.657+0000 7f6b697fa640 1 -- 172.21.15.132:0/1843429579 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c03be60 con 0x7f6b6c100e90 2024-04-23T18:35:29.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.657+0000 7f6b72813640 1 -- 172.21.15.132:0/1843429579 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6c06aaf0 con 0x7f6b6c100e90 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.659+0000 7f6b697fa640 1 -- 172.21.15.132:0/1843429579 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f6b5c03b5f0 con 0x7f6b6c100e90 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.659+0000 7f6b7560a740 1 Processor -- start 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.659+0000 7f6b7560a740 1 -- start start 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.659+0000 7f6b7560a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0588fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.659+0000 7f6b7560a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f058a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b7560a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 0x55e5f0590fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b7560a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55e5f04edc50 con 0x55e5f0589d50 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b7560a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e5f048a830 con 0x55e5f0588b80 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b7560a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e5f04e9c10 con 0x55e5f058a710 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b697fa640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6b540a3bb0 0x7f6b540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b6bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f058a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b6bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f058a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34616/0 (socket says 172.21.15.132:34616) 2024-04-23T18:35:29.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b6bfff640 1 -- 172.21.15.132:0/3736645881 learned_addr learned my addr 172.21.15.132:0/3736645881 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b70db0640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6b540a3bb0 0x7f6b540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b70db0640 1 --2- 172.21.15.132:0/3736645881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0588fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b6b7fe640 1 --2- 172.21.15.132:0/3736645881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 0x55e5f0590fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b697fa640 1 -- 172.21.15.132:0/1843429579 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6b5c0f2f70 con 0x7f6b6c100e90 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b6bfff640 1 -- 172.21.15.132:0/3736645881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 msgr2=0x55e5f0590fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/3736645881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 0x55e5f0590fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b6bfff640 1 -- 172.21.15.132:0/3736645881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 msgr2=0x55e5f0588fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/3736645881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0588fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b6bfff640 1 -- 172.21.15.132:0/3736645881 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f02002d0 con 0x55e5f0589d50 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b70db0640 1 --2- 172.21.15.132:0/3736645881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0588fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/3736645881 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f058a1d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0a8770 tx=0x7f6b5c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35ec92dda5b767a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b70db0640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6b540a3bb0 0x7f6b540a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55e5f0589c50 tx=0x7f6b640094b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b527fc640 1 -- 172.21.15.132:0/3736645881 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0bf8e0 con 0x55e5f0589d50 2024-04-23T18:35:29.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b527fc640 1 -- 172.21.15.132:0/3736645881 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b5c0c3590 con 0x55e5f0589d50 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.660+0000 7f6b527fc640 1 -- 172.21.15.132:0/3736645881 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0bb850 con 0x55e5f0589d50 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.661+0000 7f6b7560a740 1 -- 172.21.15.132:0/3736645881 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 msgr2=0x55e5f058a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.661+0000 7f6b7560a740 1 --2- 172.21.15.132:0/3736645881 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f058a1d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0a8770 tx=0x7f6b5c02a410 comp rx=0 tx=0).stop 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.661+0000 7f6b7560a740 1 -- 172.21.15.132:0/3736645881 shutdown_connections 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.661+0000 7f6b7560a740 1 --2- 172.21.15.132:0/3736645881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 0x55e5f0590fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.661+0000 7f6b7560a740 1 --2- 172.21.15.132:0/3736645881 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f058a1d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.661+0000 7f6b7560a740 1 --2- 172.21.15.132:0/3736645881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0588fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.661+0000 7f6b7560a740 1 -- 172.21.15.132:0/3736645881 >> 172.21.15.132:0/3736645881 conn(0x55e5f0584070 msgr2=0x55e5f0584470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.661+0000 7f6b7560a740 1 -- 172.21.15.132:0/3736645881 shutdown_connections 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.661+0000 7f6b7560a740 1 -- 172.21.15.132:0/3736645881 wait complete. 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.661+0000 7f6b7560a740 1 -- start start 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b7560a740 1 Processor -- start 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b7560a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f0588b80 0x55e5f0597070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b7560a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f05975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b7560a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f058a710 0x55e5f059e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b7560a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55e5f04edc50 con 0x55e5f0589d50 2024-04-23T18:35:29.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b7560a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e5f0488470 con 0x55e5f058a710 2024-04-23T18:35:29.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b7560a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e5f05914e0 con 0x55e5f0588b80 2024-04-23T18:35:29.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b7560a740 10 client.0 _ll_register_callbacks cb 0x55e5f04adf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:29.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b6b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f05975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b6b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f05975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34632/0 (socket says 172.21.15.132:34632) 2024-04-23T18:35:29.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b6b7fe640 1 -- 172.21.15.132:0/1186509486 learned_addr learned my addr 172.21.15.132:0/1186509486 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f0588b80 0x55e5f0597070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b70db0640 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f058a710 0x55e5f059e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b6b7fe640 1 -- 172.21.15.132:0/1186509486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f0588b80 msgr2=0x55e5f0597070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b6b7fe640 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f0588b80 0x55e5f0597070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b6b7fe640 1 -- 172.21.15.132:0/1186509486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f058a710 msgr2=0x55e5f059e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b6b7fe640 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f058a710 0x55e5f059e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b6b7fe640 1 -- 172.21.15.132:0/1186509486 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f02ca6b0 con 0x55e5f0589d50 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b6b7fe640 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f05975b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000f6f0 tx=0x7f6b6000fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36c2d69f885c7ff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.662+0000 7f6b4b7fe640 1 -- 172.21.15.132:0/1186509486 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60016070 con 0x55e5f0589d50 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b4b7fe640 1 -- 172.21.15.132:0/1186509486 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6b60004220 con 0x55e5f0589d50 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b7560a740 1 -- 172.21.15.132:0/1186509486 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b5c0c3590 con 0x55e5f0589d50 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b4b7fe640 1 -- 172.21.15.132:0/1186509486 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b60004420 con 0x55e5f0589d50 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b7560a740 10 client.15084 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b7560a740 1 -- 172.21.15.132:0/1186509486 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e5f05a1a30 con 0x55e5f0589d50 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b4b7fe640 1 -- 172.21.15.132:0/1186509486 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b5c0c3590 con 0x55e5f0589d50 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b4b7fe640 1 -- 172.21.15.132:0/1186509486 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6b60023050 con 0x55e5f0589d50 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b7560a740 10 client.15084 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:29.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b7560a740 10 client.15084 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b7560a740 20 client.15084 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b7560a740 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x55e5f05a40d0 0x55e5f05a5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b7560a740 4 client.15084 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b7560a740 1 -- 172.21.15.132:0/1186509486 --> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5f04fb670 con 0x55e5f05a40d0 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.663+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x55e5f05a40d0 0x55e5f05a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.664+0000 7f6b4b7fe640 10 client.15084 ms_handle_connect on v2:172.21.15.148:6832/3556073608 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.664+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x55e5f05a40d0 0x55e5f05a5540 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c9d065c7fa6816c server_cookie=ef8391c70fa4c057 in_seq=0 out_seq=0 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.664+0000 7f6b4b7fe640 1 -- 172.21.15.132:0/1186509486 <== mds.3 v2:172.21.15.148:6832/3556073608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5f04fb670 con 0x55e5f05a40d0 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.664+0000 7f6b4b7fe640 10 client.15084 handle_command_reply: tid=0 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b7560a740 1 client.15084 shutdown 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b7560a740 1 -- 172.21.15.132:0/1186509486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 msgr2=0x55e5f05975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b7560a740 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f05975b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000f6f0 tx=0x7f6b6000fbb0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b7560a740 1 -- 172.21.15.132:0/1186509486 shutdown_connections 2024-04-23T18:35:29.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b7560a740 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x55e5f05a40d0 0x55e5f05a5540 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b7560a740 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f058a710 0x55e5f059e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b4b7fe640 0 client.15084 ms_handle_reset on v2:172.21.15.148:6832/3556073608 2024-04-23T18:35:29.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b7560a740 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f05975b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b7560a740 1 --2- 172.21.15.132:0/1186509486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f0588b80 0x55e5f0597070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b7560a740 1 -- 172.21.15.132:0/1186509486 >> 172.21.15.132:0/1186509486 conn(0x55e5f0584070 msgr2=0x55e5f05831d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b7560a740 1 -- 172.21.15.132:0/1186509486 shutdown_connections 2024-04-23T18:35:29.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b7560a740 1 -- 172.21.15.132:0/1186509486 wait complete. 2024-04-23T18:35:29.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.665+0000 7f6b7560a740 20 client.15084 trim_cache size 0 max 16384 2024-04-23T18:35:29.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b7560a740 1 Processor -- start 2024-04-23T18:35:29.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b7560a740 1 -- start start 2024-04-23T18:35:29.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b7560a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0583b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b7560a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f059e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b7560a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 0x55e5f0582940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b7560a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55e5f05a3880 con 0x55e5f0589d50 2024-04-23T18:35:29.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b7560a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e5f05a3f50 con 0x55e5f0588b80 2024-04-23T18:35:29.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b7560a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e5f05a4620 con 0x55e5f058a710 2024-04-23T18:35:29.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b6b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f059e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b6b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f059e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34642/0 (socket says 172.21.15.132:34642) 2024-04-23T18:35:29.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b6b7fe640 1 -- 172.21.15.132:0/2506663198 learned_addr learned my addr 172.21.15.132:0/2506663198 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b70db0640 1 --2- 172.21.15.132:0/2506663198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 0x55e5f0582940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b6b7fe640 1 -- 172.21.15.132:0/2506663198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 msgr2=0x55e5f0582940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b6b7fe640 1 --2- 172.21.15.132:0/2506663198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 0x55e5f0582940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b6b7fe640 1 -- 172.21.15.132:0/2506663198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 msgr2=0x55e5f0583b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b6b7fe640 1 --2- 172.21.15.132:0/2506663198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0583b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b6b7fe640 1 -- 172.21.15.132:0/2506663198 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f05897d0 con 0x55e5f0589d50 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b6b7fe640 1 --2- 172.21.15.132:0/2506663198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f059e5b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000f6c0 tx=0x7f6b6000dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1901161eca4b89 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.683+0000 7f6b51ffb640 1 -- 172.21.15.132:0/2506663198 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60016070 con 0x55e5f0589d50 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.684+0000 7f6b51ffb640 1 -- 172.21.15.132:0/2506663198 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b600040c0 con 0x55e5f0589d50 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.684+0000 7f6b51ffb640 1 -- 172.21.15.132:0/2506663198 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6000b6f0 con 0x55e5f0589d50 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.684+0000 7f6b7560a740 1 -- 172.21.15.132:0/2506663198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 msgr2=0x55e5f059e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.684+0000 7f6b7560a740 1 --2- 172.21.15.132:0/2506663198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f059e5b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000f6c0 tx=0x7f6b6000dfd0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.684+0000 7f6b7560a740 1 -- 172.21.15.132:0/2506663198 shutdown_connections 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.684+0000 7f6b7560a740 1 --2- 172.21.15.132:0/2506663198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 0x55e5f0582940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.684+0000 7f6b7560a740 1 --2- 172.21.15.132:0/2506663198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f059e5b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.684+0000 7f6b7560a740 1 --2- 172.21.15.132:0/2506663198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0583b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.684+0000 7f6b7560a740 1 -- 172.21.15.132:0/2506663198 >> 172.21.15.132:0/2506663198 conn(0x55e5f0584070 msgr2=0x55e5f05956d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.684+0000 7f6b7560a740 1 -- 172.21.15.132:0/2506663198 shutdown_connections 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.684+0000 7f6b7560a740 1 -- 172.21.15.132:0/2506663198 wait complete. 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.684+0000 7f6b7560a740 1 -- start start 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b7560a740 1 Processor -- start 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b7560a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0599b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b7560a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f059a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b7560a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 0x55e5f059d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b7560a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55e5f05a53c0 con 0x55e5f0589d50 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b7560a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e5f05a4cf0 con 0x55e5f0588b80 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b7560a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e5f05a5a90 con 0x55e5f058a710 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b7560a740 10 client.0 _ll_register_callbacks cb 0x7f6b6000d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b6b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f059a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b6b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f059a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34646/0 (socket says 172.21.15.132:34646) 2024-04-23T18:35:29.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b6b7fe640 1 -- 172.21.15.132:0/2345924895 learned_addr learned my addr 172.21.15.132:0/2345924895 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b70db0640 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 0x55e5f059d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0599b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b6b7fe640 1 -- 172.21.15.132:0/2345924895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 msgr2=0x55e5f059d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b6b7fe640 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 0x55e5f059d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b6b7fe640 1 -- 172.21.15.132:0/2345924895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 msgr2=0x55e5f0599b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b6b7fe640 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0599b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b6b7fe640 1 -- 172.21.15.132:0/2345924895 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f059d690 con 0x55e5f0589d50 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0599b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b6b7fe640 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f059a080 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6b60003c50 tx=0x7f6b600038c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8140db758f8014 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b4b7fe640 1 -- 172.21.15.132:0/2345924895 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60016070 con 0x55e5f0589d50 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b4b7fe640 1 -- 172.21.15.132:0/2345924895 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6b6000b900 con 0x55e5f0589d50 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.685+0000 7f6b7560a740 1 -- 172.21.15.132:0/2345924895 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b600040c0 con 0x55e5f0589d50 2024-04-23T18:35:29.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b4b7fe640 1 -- 172.21.15.132:0/2345924895 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b6000bb20 con 0x55e5f0589d50 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b7560a740 10 client.15090 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b7560a740 1 -- 172.21.15.132:0/2345924895 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e5f059e990 con 0x55e5f0589d50 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b4b7fe640 1 -- 172.21.15.132:0/2345924895 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b600040c0 con 0x55e5f0589d50 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b4b7fe640 1 -- 172.21.15.132:0/2345924895 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6b60035030 con 0x55e5f0589d50 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b7560a740 10 client.15090 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b7560a740 10 client.15090 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b7560a740 20 client.15090 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b7560a740 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x55e5f060e630 0x55e5f060b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b7560a740 4 client.15090 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b7560a740 1 -- 172.21.15.132:0/2345924895 --> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] -- command(tid 0: {"prefix": "status"}) -- 0x55e5f04f54e0 con 0x55e5f060e630 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x55e5f060e630 0x55e5f060b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b4b7fe640 10 client.15090 ms_handle_connect on v2:172.21.15.148:6832/3556073608 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.686+0000 7f6b6bfff640 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x55e5f060e630 0x55e5f060b140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9878faa57b3d8c48 server_cookie=4aa5db39f3215fef in_seq=0 out_seq=0 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b4b7fe640 1 -- 172.21.15.132:0/2345924895 <== mds.3 v2:172.21.15.148:6832/3556073608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e5f04f54e0 con 0x55e5f060e630 2024-04-23T18:35:29.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b4b7fe640 10 client.15090 handle_command_reply: tid=0 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b7560a740 1 client.15090 shutdown 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b7560a740 1 -- 172.21.15.132:0/2345924895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 msgr2=0x55e5f059a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b7560a740 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f059a080 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6b60003c50 tx=0x7f6b600038c0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b7560a740 1 -- 172.21.15.132:0/2345924895 shutdown_connections 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b7560a740 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x55e5f060e630 0x55e5f060b140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b7560a740 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5f058a710 0x55e5f059d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b4b7fe640 0 client.15090 ms_handle_reset on v2:172.21.15.148:6832/3556073608 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b7560a740 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e5f0589d50 0x55e5f059a080 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b7560a740 1 --2- 172.21.15.132:0/2345924895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5f0588b80 0x55e5f0599b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b7560a740 1 -- 172.21.15.132:0/2345924895 >> 172.21.15.132:0/2345924895 conn(0x55e5f0584070 msgr2=0x55e5f0595b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b7560a740 1 -- 172.21.15.132:0/2345924895 shutdown_connections 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b7560a740 1 -- 172.21.15.132:0/2345924895 wait complete. 2024-04-23T18:35:29.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.687+0000 7f6b7560a740 20 client.15090 trim_cache size 0 max 16384 2024-04-23T18:35:29.687 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:35:29.687 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:35:29.687 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 3, 2024-04-23T18:35:29.687 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24425, 2024-04-23T18:35:29.687 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:35:29.687 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:35:29.687 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:35:29.687 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 84.166057217000002, 2024-04-23T18:35:29.687 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 25, 2024-04-23T18:35:29.687 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 88, 2024-04-23T18:35:29.687 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:35:29.687 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 124.052739759 2024-04-23T18:35:29.688 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:35:29.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.688+0000 7f6b72813640 1 -- 172.21.15.132:0/1843429579 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6b540a3bb0 msgr2=0x7f6b540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.688+0000 7f6b72813640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6b540a3bb0 0x7f6b540a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55e5f0589c50 tx=0x7f6b640094b0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.688+0000 7f6b72813640 1 -- 172.21.15.132:0/1843429579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b6c100e90 msgr2=0x7f6b6c1970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.688+0000 7f6b72813640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b6c100e90 0x7f6b6c1970c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c005940 tx=0x7f6b5c005e50 comp rx=0 tx=0).stop 2024-04-23T18:35:29.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.688+0000 7f6b72813640 1 -- 172.21.15.132:0/1843429579 shutdown_connections 2024-04-23T18:35:29.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.688+0000 7f6b72813640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6b540a3bb0 0x7f6b540a6070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.688+0000 7f6b72813640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6c1029a0 0x7f6b6c19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.688+0000 7f6b72813640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b6c101fe0 0x7f6b6c197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.688+0000 7f6b72813640 1 --2- 172.21.15.132:0/1843429579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b6c100e90 0x7f6b6c1970c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.688+0000 7f6b72813640 1 -- 172.21.15.132:0/1843429579 >> 172.21.15.132:0/1843429579 conn(0x7f6b6c0fc400 msgr2=0x7f6b6c105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.688+0000 7f6b72813640 1 -- 172.21.15.132:0/1843429579 shutdown_connections 2024-04-23T18:35:29.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.688+0000 7f6b72813640 1 -- 172.21.15.132:0/1843429579 wait complete. 2024-04-23T18:35:29.691 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T18:35:29.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.797+0000 7f9fafa33640 1 -- 172.21.15.132:0/2093896408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa8102a10 msgr2=0x7f9fa8104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.797+0000 7f9fafa33640 1 --2- 172.21.15.132:0/2093896408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa8102a10 0x7f9fa8104e00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009900 tx=0x7f9fa402a410 comp rx=0 tx=0).stop 2024-04-23T18:35:29.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.797+0000 7f9fafa33640 1 -- 172.21.15.132:0/2093896408 shutdown_connections 2024-04-23T18:35:29.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.797+0000 7f9fafa33640 1 --2- 172.21.15.132:0/2093896408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fa8107c70 0x7f9fa810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.797+0000 7f9fafa33640 1 --2- 172.21.15.132:0/2093896408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa8105340 0x7f9fa8107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.797+0000 7f9fafa33640 1 --2- 172.21.15.132:0/2093896408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa8102a10 0x7f9fa8104e00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.797+0000 7f9fafa33640 1 -- 172.21.15.132:0/2093896408 >> 172.21.15.132:0/2093896408 conn(0x7f9fa80fc400 msgr2=0x7f9fa80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.797+0000 7f9fafa33640 1 -- 172.21.15.132:0/2093896408 shutdown_connections 2024-04-23T18:35:29.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.797+0000 7f9fafa33640 1 -- 172.21.15.132:0/2093896408 wait complete. 2024-04-23T18:35:29.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fafa33640 1 Processor -- start 2024-04-23T18:35:29.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fafa33640 1 -- start start 2024-04-23T18:35:29.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fafa33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fa8102a10 0x7f9fa8198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fafa33640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa8105340 0x7f9fa81987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fafa33640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa8107c70 0x7f9fa819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fafa33640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa810dae0 con 0x7f9fa8107c70 2024-04-23T18:35:29.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fafa33640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa810d960 con 0x7f9fa8105340 2024-04-23T18:35:29.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fafa33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa810dc60 con 0x7f9fa8102a10 2024-04-23T18:35:29.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fadfd0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa8107c70 0x7f9fa819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fadfd0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa8107c70 0x7f9fa819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34668/0 (socket says 172.21.15.132:34668) 2024-04-23T18:35:29.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fadfd0640 1 -- 172.21.15.132:0/1451232774 learned_addr learned my addr 172.21.15.132:0/1451232774 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9facfce640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa8105340 0x7f9fa81987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fadfd0640 1 -- 172.21.15.132:0/1451232774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fa8102a10 msgr2=0x7f9fa8198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fad7cf640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fa8102a10 0x7f9fa8198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fadfd0640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fa8102a10 0x7f9fa8198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fadfd0640 1 -- 172.21.15.132:0/1451232774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa8105340 msgr2=0x7f9fa81987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fadfd0640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa8105340 0x7f9fa81987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fadfd0640 1 -- 172.21.15.132:0/1451232774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa819fe30 con 0x7f9fa8107c70 2024-04-23T18:35:29.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9facfce640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa8105340 0x7f9fa81987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:29.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fadfd0640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa8107c70 0x7f9fa819f7c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000cad0 tx=0x7f9fa000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9f9e7fc640 1 -- 172.21.15.132:0/1451232774 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa000ae20 con 0x7f9fa8107c70 2024-04-23T18:35:29.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9f9e7fc640 1 -- 172.21.15.132:0/1451232774 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fa0005020 con 0x7f9fa8107c70 2024-04-23T18:35:29.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9fafa33640 1 -- 172.21.15.132:0/1451232774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa81a0120 con 0x7f9fa8107c70 2024-04-23T18:35:29.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.798+0000 7f9f9e7fc640 1 -- 172.21.15.132:0/1451232774 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa000f6b0 con 0x7f9fa8107c70 2024-04-23T18:35:29.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.799+0000 7f9fafa33640 1 -- 172.21.15.132:0/1451232774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa81a0670 con 0x7f9fa8107c70 2024-04-23T18:35:29.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9f9e7fc640 1 -- 172.21.15.132:0/1451232774 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f9fa00051d0 con 0x7f9fa8107c70 2024-04-23T18:35:29.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9fb2820740 1 Processor -- start 2024-04-23T18:35:29.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9fb2820740 1 -- start start 2024-04-23T18:35:29.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9fb2820740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c67f90 0x55f186c683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9fb2820740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 0x55f186c695e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9fb2820740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c69b20 0x55f186c703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9fb2820740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55f186bc8c50 con 0x55f186c67f90 2024-04-23T18:35:29.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9fb2820740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f186b65830 con 0x55f186c69b20 2024-04-23T18:35:29.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9fb2820740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f186bc4c10 con 0x55f186c69160 2024-04-23T18:35:29.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9f9e7fc640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9f940a3bb0 0x7f9f940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9facfce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c67f90 0x55f186c683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9facfce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c67f90 0x55f186c683b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34682/0 (socket says 172.21.15.132:34682) 2024-04-23T18:35:29.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.801+0000 7f9facfce640 1 -- 172.21.15.132:0/3557003786 learned_addr learned my addr 172.21.15.132:0/3557003786 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9fadfd0640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9f940a3bb0 0x7f9f940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9fad7cf640 1 --2- 172.21.15.132:0/3557003786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 0x55f186c695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9facfce640 1 -- 172.21.15.132:0/3557003786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 msgr2=0x55f186c695e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9facfce640 1 --2- 172.21.15.132:0/3557003786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 0x55f186c695e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9f9e7fc640 1 -- 172.21.15.132:0/1451232774 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9fa00d04d0 con 0x7f9fa8107c70 2024-04-23T18:35:29.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9facfce640 1 -- 172.21.15.132:0/3557003786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c69b20 msgr2=0x55f186c703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9facfce640 1 --2- 172.21.15.132:0/3557003786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c69b20 0x55f186c703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9facfce640 1 -- 172.21.15.132:0/3557003786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1868db2d0 con 0x55f186c67f90 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9fad7cf640 1 --2- 172.21.15.132:0/3557003786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 0x55f186c695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9facfce640 1 --2- 172.21.15.132:0/3557003786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c67f90 0x55f186c683b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009c80 tx=0x7f9f980106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40d02f9344c5518 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9fadfd0640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9f940a3bb0 0x7f9f940a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0098c00 tx=0x7f9fa006b1a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9f937fe640 1 -- 172.21.15.132:0/3557003786 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98016070 con 0x55f186c67f90 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9f937fe640 1 -- 172.21.15.132:0/3557003786 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f980128d0 con 0x55f186c67f90 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.802+0000 7f9f937fe640 1 -- 172.21.15.132:0/3557003786 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98011760 con 0x55f186c67f90 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.803+0000 7f9fb2820740 1 -- 172.21.15.132:0/3557003786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c67f90 msgr2=0x55f186c683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.803+0000 7f9fb2820740 1 --2- 172.21.15.132:0/3557003786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c67f90 0x55f186c683b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009c80 tx=0x7f9f980106e0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.803+0000 7f9fb2820740 1 -- 172.21.15.132:0/3557003786 shutdown_connections 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.803+0000 7f9fb2820740 1 --2- 172.21.15.132:0/3557003786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 0x55f186c695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.803+0000 7f9fb2820740 1 --2- 172.21.15.132:0/3557003786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c69b20 0x55f186c703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.803+0000 7f9fb2820740 1 --2- 172.21.15.132:0/3557003786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c67f90 0x55f186c683b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.803+0000 7f9fb2820740 1 -- 172.21.15.132:0/3557003786 >> 172.21.15.132:0/3557003786 conn(0x55f186c5f070 msgr2=0x55f186c5f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.803+0000 7f9fb2820740 1 -- 172.21.15.132:0/3557003786 shutdown_connections 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.803+0000 7f9fb2820740 1 -- 172.21.15.132:0/3557003786 wait complete. 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.803+0000 7f9fb2820740 1 -- start start 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.803+0000 7f9fb2820740 1 Processor -- start 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9fb2820740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c72440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9fb2820740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 0x55f186c72980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9fb2820740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 0x55f186c799c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9fb2820740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55f186c70b30 con 0x55f186c69b20 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9fb2820740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f186b63470 con 0x55f186c67f90 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9fb2820740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f186c70cb0 con 0x55f186c69160 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9fb2820740 10 client.0 _ll_register_callbacks cb 0x55f186b88f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9facfce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 0x55f186c799c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9facfce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 0x55f186c799c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34696/0 (socket says 172.21.15.132:34696) 2024-04-23T18:35:29.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9facfce640 1 -- 172.21.15.132:0/1031319930 learned_addr learned my addr 172.21.15.132:0/1031319930 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9facfce640 1 --2- 172.21.15.132:0/1031319930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c72440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9facfce640 1 -- 172.21.15.132:0/1031319930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 msgr2=0x55f186c72980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9facfce640 1 --2- 172.21.15.132:0/1031319930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 0x55f186c72980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9facfce640 1 -- 172.21.15.132:0/1031319930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 msgr2=0x55f186c72440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9facfce640 1 --2- 172.21.15.132:0/1031319930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c72440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9facfce640 1 -- 172.21.15.132:0/1031319930 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1869a56b0 con 0x55f186c69b20 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9facfce640 1 --2- 172.21.15.132:0/1031319930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 0x55f186c799c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9f980095a0 tx=0x7f9f98009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a21679ec5aebdd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9f90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9f90ff9640 1 -- 172.21.15.132:0/1031319930 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9801f070 con 0x55f186c69b20 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9f90ff9640 1 -- 172.21.15.132:0/1031319930 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9f980126b0 con 0x55f186c69b20 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9fb2820740 1 -- 172.21.15.132:0/1031319930 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f980128d0 con 0x55f186c69b20 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9f90ff9640 1 -- 172.21.15.132:0/1031319930 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f98016030 con 0x55f186c69b20 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9fb2820740 10 client.15102 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:29.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.804+0000 7f9fb2820740 1 -- 172.21.15.132:0/1031319930 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f186c7cda0 con 0x55f186c69b20 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.805+0000 7f9f90ff9640 1 -- 172.21.15.132:0/1031319930 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f980128d0 con 0x55f186c69b20 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.805+0000 7f9f90ff9640 1 -- 172.21.15.132:0/1031319930 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9f9801b4e0 con 0x55f186c69b20 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.805+0000 7f9fb2820740 10 client.15102 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.805+0000 7f9fb2820740 10 client.15102 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.805+0000 7f9fb2820740 20 client.15102 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.805+0000 7f9fb2820740 1 --2- 172.21.15.132:0/1031319930 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x55f186c7f190 0x55f186c80600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.805+0000 7f9fb2820740 4 client.15102 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.805+0000 7f9fb2820740 1 -- 172.21.15.132:0/1031319930 --> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f186bd6670 con 0x55f186c7f190 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.805+0000 7f9fad7cf640 1 --2- 172.21.15.132:0/1031319930 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x55f186c7f190 0x55f186c80600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.805+0000 7f9f90ff9640 10 client.15102 ms_handle_connect on v2:172.21.15.157:6838/1528717147 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.805+0000 7f9fad7cf640 1 --2- 172.21.15.132:0/1031319930 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x55f186c7f190 0x55f186c80600 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67fd68143b8404b1 server_cookie=31203571aeac0f3 in_seq=0 out_seq=0 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.806+0000 7f9f90ff9640 1 -- 172.21.15.132:0/1031319930 <== mds.2 v2:172.21.15.157:6838/1528717147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f186bd6670 con 0x55f186c7f190 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.806+0000 7f9f90ff9640 10 client.15102 handle_command_reply: tid=0 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.806+0000 7f9fb2820740 1 client.15102 shutdown 2024-04-23T18:35:29.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.806+0000 7f9fb2820740 1 -- 172.21.15.132:0/1031319930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 msgr2=0x55f186c799c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.806+0000 7f9fb2820740 1 --2- 172.21.15.132:0/1031319930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 0x55f186c799c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9f980095a0 tx=0x7f9f98009530 comp rx=0 tx=0).stop 2024-04-23T18:35:29.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.806+0000 7f9fb2820740 1 -- 172.21.15.132:0/1031319930 shutdown_connections 2024-04-23T18:35:29.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.806+0000 7f9fb2820740 1 --2- 172.21.15.132:0/1031319930 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x55f186c7f190 0x55f186c80600 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.806+0000 7f9fb2820740 1 --2- 172.21.15.132:0/1031319930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 0x55f186c72980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.806+0000 7f9f90ff9640 0 client.15102 ms_handle_reset on v2:172.21.15.157:6838/1528717147 2024-04-23T18:35:29.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.806+0000 7f9fb2820740 1 --2- 172.21.15.132:0/1031319930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 0x55f186c799c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.806+0000 7f9fb2820740 1 --2- 172.21.15.132:0/1031319930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c72440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.806+0000 7f9fb2820740 1 -- 172.21.15.132:0/1031319930 >> 172.21.15.132:0/1031319930 conn(0x55f186c5f070 msgr2=0x55f186c5e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.807+0000 7f9fb2820740 1 -- 172.21.15.132:0/1031319930 shutdown_connections 2024-04-23T18:35:29.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.807+0000 7f9fb2820740 1 -- 172.21.15.132:0/1031319930 wait complete. 2024-04-23T18:35:29.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.807+0000 7f9fb2820740 20 client.15102 trim_cache size 0 max 16384 2024-04-23T18:35:29.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.824+0000 7f9fb2820740 1 Processor -- start 2024-04-23T18:35:29.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.824+0000 7f9fb2820740 1 -- start start 2024-04-23T18:35:29.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.824+0000 7f9fb2820740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.824+0000 7f9fb2820740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69160 0x55f186c71050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.824+0000 7f9fb2820740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69b20 0x55f186c71590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.824+0000 7f9fb2820740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55f186c7e940 con 0x55f186c69160 2024-04-23T18:35:29.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.824+0000 7f9fb2820740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f186c7f010 con 0x55f186c67f90 2024-04-23T18:35:29.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.824+0000 7f9fb2820740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f186c7f6e0 con 0x55f186c69b20 2024-04-23T18:35:29.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.824+0000 7f9fad7cf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69160 0x55f186c71050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9fad7cf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69160 0x55f186c71050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34708/0 (socket says 172.21.15.132:34708) 2024-04-23T18:35:29.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9fad7cf640 1 -- 172.21.15.132:0/463389398 learned_addr learned my addr 172.21.15.132:0/463389398 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9facfce640 1 --2- 172.21.15.132:0/463389398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9fad7cf640 1 -- 172.21.15.132:0/463389398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69b20 msgr2=0x55f186c71590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9fad7cf640 1 --2- 172.21.15.132:0/463389398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69b20 0x55f186c71590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9fad7cf640 1 -- 172.21.15.132:0/463389398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 msgr2=0x55f186c728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9fad7cf640 1 --2- 172.21.15.132:0/463389398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9fad7cf640 1 -- 172.21.15.132:0/463389398 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f186c6ff30 con 0x55f186c69160 2024-04-23T18:35:29.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9facfce640 1 --2- 172.21.15.132:0/463389398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:29.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9fad7cf640 1 --2- 172.21.15.132:0/463389398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69160 0x55f186c71050 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009230 tx=0x7f9fa4032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e87af5ac2079c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9f92ffd640 1 -- 172.21.15.132:0/463389398 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4035040 con 0x55f186c69160 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9f92ffd640 1 -- 172.21.15.132:0/463389398 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fa4045020 con 0x55f186c69160 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.825+0000 7f9f92ffd640 1 -- 172.21.15.132:0/463389398 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40046c0 con 0x55f186c69160 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 -- 172.21.15.132:0/463389398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69160 msgr2=0x55f186c71050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 --2- 172.21.15.132:0/463389398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69160 0x55f186c71050 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009230 tx=0x7f9fa4032d50 comp rx=0 tx=0).stop 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 -- 172.21.15.132:0/463389398 shutdown_connections 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 --2- 172.21.15.132:0/463389398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69b20 0x55f186c71590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 --2- 172.21.15.132:0/463389398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69160 0x55f186c71050 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 --2- 172.21.15.132:0/463389398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 -- 172.21.15.132:0/463389398 >> 172.21.15.132:0/463389398 conn(0x55f186c5f070 msgr2=0x55f186c5a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 -- 172.21.15.132:0/463389398 shutdown_connections 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 -- 172.21.15.132:0/463389398 wait complete. 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 -- start start 2024-04-23T18:35:29.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 Processor -- start 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 0x55f186c75e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 0x55f186c78350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55f186c80480 con 0x55f186c69b20 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f186c7fdb0 con 0x55f186c67f90 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f186c80b50 con 0x55f186c69160 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9fb2820740 10 client.0 _ll_register_callbacks cb 0x7f9f98009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9facfce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 0x55f186c78350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9facfce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 0x55f186c78350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34718/0 (socket says 172.21.15.132:34718) 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.826+0000 7f9facfce640 1 -- 172.21.15.132:0/1335621377 learned_addr learned my addr 172.21.15.132:0/1335621377 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9facfce640 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9fad7cf640 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 0x55f186c75e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9facfce640 1 -- 172.21.15.132:0/1335621377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 msgr2=0x55f186c75e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9facfce640 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 0x55f186c75e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9facfce640 1 -- 172.21.15.132:0/1335621377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 msgr2=0x55f186c738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9facfce640 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9facfce640 1 -- 172.21.15.132:0/1335621377 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f186c78890 con 0x55f186c69b20 2024-04-23T18:35:29.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9facfce640 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 0x55f186c78350 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9f9801b900 tx=0x7f9f98010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454d57d81c503a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9f90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:29.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9f90ff9640 1 -- 172.21.15.132:0/1335621377 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9801f040 con 0x55f186c69b20 2024-04-23T18:35:29.829 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9fb2820740 1 -- 172.21.15.132:0/1335621377 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa4045020 con 0x55f186c69b20 2024-04-23T18:35:29.829 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9f90ff9640 1 -- 172.21.15.132:0/1335621377 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f9f9801b450 con 0x55f186c69b20 2024-04-23T18:35:29.829 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9f90ff9640 1 -- 172.21.15.132:0/1335621377 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f9803f050 con 0x55f186c69b20 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9fb2820740 10 client.15108 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9fb2820740 1 -- 172.21.15.132:0/1335621377 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f186c79b50 con 0x55f186c69b20 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9f90ff9640 1 -- 172.21.15.132:0/1335621377 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fa4045020 con 0x55f186c69b20 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9f90ff9640 1 -- 172.21.15.132:0/1335621377 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9f98011af0 con 0x55f186c69b20 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9fb2820740 10 client.15108 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9fb2820740 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9fb2820740 20 client.15108 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9fb2820740 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x55f186cee0b0 0x55f186ceabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9fb2820740 4 client.15108 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.827+0000 7f9fb2820740 1 -- 172.21.15.132:0/1335621377 --> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] -- command(tid 0: {"prefix": "status"}) -- 0x55f186bd04e0 con 0x55f186cee0b0 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.828+0000 7f9fad7cf640 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x55f186cee0b0 0x55f186ceabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.828+0000 7f9f90ff9640 10 client.15108 ms_handle_connect on v2:172.21.15.157:6838/1528717147 2024-04-23T18:35:29.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.828+0000 7f9fad7cf640 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x55f186cee0b0 0x55f186ceabc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=206db2cb0234343e server_cookie=12eb8200fb9a6c59 in_seq=0 out_seq=0 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.828+0000 7f9f90ff9640 1 -- 172.21.15.132:0/1335621377 <== mds.2 v2:172.21.15.157:6838/1528717147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f186bd04e0 con 0x55f186cee0b0 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.828+0000 7f9f90ff9640 10 client.15108 handle_command_reply: tid=0 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.828+0000 7f9fb2820740 1 client.15108 shutdown 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.828+0000 7f9fb2820740 1 -- 172.21.15.132:0/1335621377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 msgr2=0x55f186c78350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.828+0000 7f9fb2820740 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 0x55f186c78350 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9f9801b900 tx=0x7f9f98010f10 comp rx=0 tx=0).stop 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.828+0000 7f9fb2820740 1 -- 172.21.15.132:0/1335621377 shutdown_connections 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.828+0000 7f9fb2820740 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x55f186cee0b0 0x55f186ceabc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fb2820740 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f186c69160 0x55f186c75e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9f90ff9640 0 client.15108 ms_handle_reset on v2:172.21.15.157:6838/1528717147 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fb2820740 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f186c69b20 0x55f186c78350 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fb2820740 1 --2- 172.21.15.132:0/1335621377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f186c67f90 0x55f186c738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fb2820740 1 -- 172.21.15.132:0/1335621377 >> 172.21.15.132:0/1335621377 conn(0x55f186c5f070 msgr2=0x55f186c71050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.831 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fb2820740 1 -- 172.21.15.132:0/1335621377 shutdown_connections 2024-04-23T18:35:29.832 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fb2820740 1 -- 172.21.15.132:0/1335621377 wait complete. 2024-04-23T18:35:29.832 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fb2820740 20 client.15108 trim_cache size 0 max 16384 2024-04-23T18:35:29.832 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:35:29.832 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:35:29.832 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 2, 2024-04-23T18:35:29.832 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24460, 2024-04-23T18:35:29.832 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:35:29.832 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:35:29.832 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:35:29.832 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 85.288954806999996, 2024-04-23T18:35:29.833 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 25, 2024-04-23T18:35:29.833 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 88, 2024-04-23T18:35:29.833 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:35:29.833 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 103.989142104 2024-04-23T18:35:29.833 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:35:29.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fafa33640 1 -- 172.21.15.132:0/1451232774 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9f940a3bb0 msgr2=0x7f9f940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fafa33640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9f940a3bb0 0x7f9f940a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0098c00 tx=0x7f9fa006b1a0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fafa33640 1 -- 172.21.15.132:0/1451232774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa8107c70 msgr2=0x7f9fa819f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fafa33640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa8107c70 0x7f9fa819f7c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000cad0 tx=0x7f9fa000cf90 comp rx=0 tx=0).stop 2024-04-23T18:35:29.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fafa33640 1 -- 172.21.15.132:0/1451232774 shutdown_connections 2024-04-23T18:35:29.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fafa33640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f9f940a3bb0 0x7f9f940a6070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fafa33640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa8107c70 0x7f9fa819f7c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fafa33640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa8105340 0x7f9fa81987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fafa33640 1 --2- 172.21.15.132:0/1451232774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fa8102a10 0x7f9fa8198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fafa33640 1 -- 172.21.15.132:0/1451232774 >> 172.21.15.132:0/1451232774 conn(0x7f9fa80fc400 msgr2=0x7f9fa8103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fafa33640 1 -- 172.21.15.132:0/1451232774 shutdown_connections 2024-04-23T18:35:29.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.829+0000 7f9fafa33640 1 -- 172.21.15.132:0/1451232774 wait complete. 2024-04-23T18:35:29.835 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T18:35:29.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.942+0000 7f7b16831640 1 -- 172.21.15.132:0/2532877971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b10102a10 msgr2=0x7f7b10104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.942+0000 7f7b16831640 1 --2- 172.21.15.132:0/2532877971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b10102a10 0x7f7b10104e00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009900 tx=0x7f7b0002a410 comp rx=0 tx=0).stop 2024-04-23T18:35:29.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.942+0000 7f7b16831640 1 -- 172.21.15.132:0/2532877971 shutdown_connections 2024-04-23T18:35:29.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.942+0000 7f7b16831640 1 --2- 172.21.15.132:0/2532877971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b10105340 0x7f7b10107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.942+0000 7f7b16831640 1 --2- 172.21.15.132:0/2532877971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b10107c70 0x7f7b1010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.942+0000 7f7b16831640 1 --2- 172.21.15.132:0/2532877971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b10102a10 0x7f7b10104e00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.942+0000 7f7b16831640 1 -- 172.21.15.132:0/2532877971 >> 172.21.15.132:0/2532877971 conn(0x7f7b100fc400 msgr2=0x7f7b100fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.942+0000 7f7b16831640 1 -- 172.21.15.132:0/2532877971 shutdown_connections 2024-04-23T18:35:29.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.942+0000 7f7b16831640 1 -- 172.21.15.132:0/2532877971 wait complete. 2024-04-23T18:35:29.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b16831640 1 Processor -- start 2024-04-23T18:35:29.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b16831640 1 -- start start 2024-04-23T18:35:29.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b16831640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b10102a10 0x7f7b10198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b16831640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b10105340 0x7f7b101987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b16831640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b10107c70 0x7f7b1019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b16831640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1010db40 con 0x7f7b10102a10 2024-04-23T18:35:29.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b16831640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1010d9c0 con 0x7f7b10105340 2024-04-23T18:35:29.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b16831640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1010dcc0 con 0x7f7b10107c70 2024-04-23T18:35:29.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b0ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b10102a10 0x7f7b10198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b0ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b10102a10 0x7f7b10198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34748/0 (socket says 172.21.15.132:34748) 2024-04-23T18:35:29.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b0ffff640 1 -- 172.21.15.132:0/2343637086 learned_addr learned my addr 172.21.15.132:0/2343637086 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b14dce640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b10107c70 0x7f7b1019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b0f7fe640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b10105340 0x7f7b101987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b0ffff640 1 -- 172.21.15.132:0/2343637086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b10107c70 msgr2=0x7f7b1019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b0ffff640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b10107c70 0x7f7b1019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b0ffff640 1 -- 172.21.15.132:0/2343637086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b10105340 msgr2=0x7f7b101987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.943+0000 7f7b0ffff640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b10105340 0x7f7b101987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.944+0000 7f7b0ffff640 1 -- 172.21.15.132:0/2343637086 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1019fe90 con 0x7f7b10102a10 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.944+0000 7f7b0f7fe640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b10105340 0x7f7b101987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.944+0000 7f7b0ffff640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b10102a10 0x7f7b10198270 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7b00005940 tx=0x7f7b00004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.944+0000 7f7b0d7fa640 1 -- 172.21.15.132:0/2343637086 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0003f030 con 0x7f7b10102a10 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.944+0000 7f7b0d7fa640 1 -- 172.21.15.132:0/2343637086 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b00003cb0 con 0x7f7b10102a10 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.944+0000 7f7b16831640 1 -- 172.21.15.132:0/2343637086 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b101a0120 con 0x7f7b10102a10 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.944+0000 7f7b0d7fa640 1 -- 172.21.15.132:0/2343637086 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00035070 con 0x7f7b10102a10 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.944+0000 7f7b16831640 1 -- 172.21.15.132:0/2343637086 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b101a0690 con 0x7f7b10102a10 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.946+0000 7f7b0d7fa640 1 -- 172.21.15.132:0/2343637086 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f7b00042030 con 0x7f7b10102a10 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.946+0000 7f7b19621740 1 Processor -- start 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.946+0000 7f7b19621740 1 -- start start 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b19621740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc4fb80 0x557c7bc4ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b19621740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc50d50 0x557c7bc511d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b19621740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 0x557c7bc57fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b19621740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x557c7bbb4c50 con 0x557c7bc51710 2024-04-23T18:35:29.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b19621740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557c7bb51830 con 0x557c7bc4fb80 2024-04-23T18:35:29.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b19621740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557c7bbb0c10 con 0x557c7bc50d50 2024-04-23T18:35:29.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b0d7fa640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7af80a3bb0 0x7f7af80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b0f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 0x557c7bc57fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b0f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 0x557c7bc57fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34756/0 (socket says 172.21.15.132:34756) 2024-04-23T18:35:29.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b0f7fe640 1 -- 172.21.15.132:0/247870612 learned_addr learned my addr 172.21.15.132:0/247870612 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b14dce640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7af80a3bb0 0x7f7af80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b0ffff640 1 --2- 172.21.15.132:0/247870612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc50d50 0x557c7bc511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b14dce640 1 --2- 172.21.15.132:0/247870612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc4fb80 0x557c7bc4ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b0d7fa640 1 -- 172.21.15.132:0/2343637086 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7b000f0d60 con 0x7f7b10102a10 2024-04-23T18:35:29.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b0f7fe640 1 -- 172.21.15.132:0/247870612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc50d50 msgr2=0x557c7bc511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b0f7fe640 1 --2- 172.21.15.132:0/247870612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc50d50 0x557c7bc511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b0f7fe640 1 -- 172.21.15.132:0/247870612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc4fb80 msgr2=0x557c7bc4ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b0f7fe640 1 --2- 172.21.15.132:0/247870612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc4fb80 0x557c7bc4ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b0f7fe640 1 -- 172.21.15.132:0/247870612 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c7b8c72d0 con 0x557c7bc51710 2024-04-23T18:35:29.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b14dce640 1 --2- 172.21.15.132:0/247870612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc4fb80 0x557c7bc4ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:29.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b0f7fe640 1 --2- 172.21.15.132:0/247870612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 0x557c7bc57fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400bce0 tx=0x7f7b0400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66c9fdbd1379880 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7b14dce640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7af80a3bb0 0x7f7af80a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x557c7bc50c50 tx=0x7f7b08009c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7af67fc640 1 -- 172.21.15.132:0/247870612 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04015070 con 0x557c7bc51710 2024-04-23T18:35:29.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7af67fc640 1 -- 172.21.15.132:0/247870612 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b0401a910 con 0x557c7bc51710 2024-04-23T18:35:29.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.947+0000 7f7af67fc640 1 -- 172.21.15.132:0/247870612 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04011910 con 0x557c7bc51710 2024-04-23T18:35:29.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.948+0000 7f7b19621740 1 -- 172.21.15.132:0/247870612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 msgr2=0x557c7bc57fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.948+0000 7f7b19621740 1 --2- 172.21.15.132:0/247870612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 0x557c7bc57fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400bce0 tx=0x7f7b0400f730 comp rx=0 tx=0).stop 2024-04-23T18:35:29.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.948+0000 7f7b19621740 1 -- 172.21.15.132:0/247870612 shutdown_connections 2024-04-23T18:35:29.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.948+0000 7f7b19621740 1 --2- 172.21.15.132:0/247870612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc50d50 0x557c7bc511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.948+0000 7f7b19621740 1 --2- 172.21.15.132:0/247870612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 0x557c7bc57fa0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.948+0000 7f7b19621740 1 --2- 172.21.15.132:0/247870612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc4fb80 0x557c7bc4ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.948+0000 7f7b19621740 1 -- 172.21.15.132:0/247870612 >> 172.21.15.132:0/247870612 conn(0x557c7bc4b070 msgr2=0x557c7bc4b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.948+0000 7f7b19621740 1 -- 172.21.15.132:0/247870612 shutdown_connections 2024-04-23T18:35:29.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.948+0000 7f7b19621740 1 -- 172.21.15.132:0/247870612 wait complete. 2024-04-23T18:35:29.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.948+0000 7f7b19621740 1 -- start start 2024-04-23T18:35:29.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b19621740 1 Processor -- start 2024-04-23T18:35:29.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b19621740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc4fb80 0x557c7bc5a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b19621740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 0x557c7bc5a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b19621740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc51710 0x557c7bc61580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b19621740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x557c7bc58720 con 0x557c7bc50d50 2024-04-23T18:35:29.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b19621740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557c7bb4f470 con 0x557c7bc4fb80 2024-04-23T18:35:29.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b19621740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557c7bc588a0 con 0x557c7bc51710 2024-04-23T18:35:29.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b19621740 10 client.0 _ll_register_callbacks cb 0x557c7bb74f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:29.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b0f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 0x557c7bc5a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b0f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 0x557c7bc5a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34764/0 (socket says 172.21.15.132:34764) 2024-04-23T18:35:29.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b0f7fe640 1 -- 172.21.15.132:0/282886796 learned_addr learned my addr 172.21.15.132:0/282886796 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b0ffff640 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc4fb80 0x557c7bc5a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b14dce640 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc51710 0x557c7bc61580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b0f7fe640 1 -- 172.21.15.132:0/282886796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc51710 msgr2=0x557c7bc61580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b0f7fe640 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc51710 0x557c7bc61580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b0f7fe640 1 -- 172.21.15.132:0/282886796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc4fb80 msgr2=0x557c7bc5a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b0f7fe640 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc4fb80 0x557c7bc5a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b0f7fe640 1 -- 172.21.15.132:0/282886796 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c7b9916b0 con 0x557c7bc50d50 2024-04-23T18:35:29.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b0ffff640 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc4fb80 0x557c7bc5a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:29.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b14dce640 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc51710 0x557c7bc61580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:29.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b0f7fe640 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 0x557c7bc5a540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7b04004630 tx=0x7f7b0401a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be66fc36bcd7dd65 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7aef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:29.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7aef7fe640 1 -- 172.21.15.132:0/282886796 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04015070 con 0x557c7bc50d50 2024-04-23T18:35:29.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7aef7fe640 1 -- 172.21.15.132:0/282886796 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7b0401aac0 con 0x557c7bc50d50 2024-04-23T18:35:29.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.949+0000 7f7b19621740 1 -- 172.21.15.132:0/282886796 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0401a910 con 0x557c7bc50d50 2024-04-23T18:35:29.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.950+0000 7f7aef7fe640 1 -- 172.21.15.132:0/282886796 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7b0401acc0 con 0x557c7bc50d50 2024-04-23T18:35:29.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.950+0000 7f7b19621740 10 client.15120 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:29.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.950+0000 7f7b19621740 1 -- 172.21.15.132:0/282886796 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c7bc648f0 con 0x557c7bc50d50 2024-04-23T18:35:29.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.950+0000 7f7aef7fe640 1 -- 172.21.15.132:0/282886796 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b0401a910 con 0x557c7bc50d50 2024-04-23T18:35:29.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.950+0000 7f7aef7fe640 1 -- 172.21.15.132:0/282886796 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7b0402d050 con 0x557c7bc50d50 2024-04-23T18:35:29.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.950+0000 7f7b19621740 10 client.15120 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:29.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.950+0000 7f7b19621740 10 client.15120 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:35:29.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.950+0000 7f7b19621740 20 client.15120 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:29.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.950+0000 7f7b19621740 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x557c7bc66d10 0x557c7bc68180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.950+0000 7f7b19621740 4 client.15120 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:29.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.950+0000 7f7b19621740 1 -- 172.21.15.132:0/282886796 --> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c7bbc2670 con 0x557c7bc66d10 2024-04-23T18:35:29.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.950+0000 7f7b0ffff640 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x557c7bc66d10 0x557c7bc68180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.951+0000 7f7aef7fe640 10 client.15120 ms_handle_connect on v2:172.21.15.148:6836/2626530887 2024-04-23T18:35:29.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.951+0000 7f7b0ffff640 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x557c7bc66d10 0x557c7bc68180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e98189f6c91801c server_cookie=d26445eba0c720df in_seq=0 out_seq=0 2024-04-23T18:35:29.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.951+0000 7f7aef7fe640 1 -- 172.21.15.132:0/282886796 <== mds.1 v2:172.21.15.148:6836/2626530887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c7bbc2670 con 0x557c7bc66d10 2024-04-23T18:35:29.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.951+0000 7f7aef7fe640 10 client.15120 handle_command_reply: tid=0 2024-04-23T18:35:29.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.951+0000 7f7b19621740 1 client.15120 shutdown 2024-04-23T18:35:29.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.952+0000 7f7b19621740 1 -- 172.21.15.132:0/282886796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 msgr2=0x557c7bc5a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.952+0000 7f7b19621740 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 0x557c7bc5a540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7b04004630 tx=0x7f7b0401a480 comp rx=0 tx=0).stop 2024-04-23T18:35:29.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.952+0000 7f7b19621740 1 -- 172.21.15.132:0/282886796 shutdown_connections 2024-04-23T18:35:29.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.952+0000 7f7b19621740 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x557c7bc66d10 0x557c7bc68180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.952+0000 7f7b19621740 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc51710 0x557c7bc61580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.952+0000 7f7b19621740 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 0x557c7bc5a540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.952+0000 7f7aef7fe640 0 client.15120 ms_handle_reset on v2:172.21.15.148:6836/2626530887 2024-04-23T18:35:29.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.952+0000 7f7b19621740 1 --2- 172.21.15.132:0/282886796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc4fb80 0x557c7bc5a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.952+0000 7f7b19621740 1 -- 172.21.15.132:0/282886796 >> 172.21.15.132:0/282886796 conn(0x557c7bc4b070 msgr2=0x557c7bc4a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.952+0000 7f7b19621740 1 -- 172.21.15.132:0/282886796 shutdown_connections 2024-04-23T18:35:29.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.952+0000 7f7b19621740 1 -- 172.21.15.132:0/282886796 wait complete. 2024-04-23T18:35:29.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.952+0000 7f7b19621740 20 client.15120 trim_cache size 0 max 16384 2024-04-23T18:35:29.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b19621740 1 Processor -- start 2024-04-23T18:35:29.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b19621740 1 -- start start 2024-04-23T18:35:29.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b19621740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc4fb80 0x557c7bc460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b19621740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 0x557c7bc61540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b19621740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc51710 0x557c7bc49940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b19621740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x557c7bc664c0 con 0x557c7bc50d50 2024-04-23T18:35:29.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b19621740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557c7bc66b90 con 0x557c7bc51710 2024-04-23T18:35:29.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b19621740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557c7bc67260 con 0x557c7bc4fb80 2024-04-23T18:35:29.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b0f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 0x557c7bc61540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b0f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 0x557c7bc61540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34780/0 (socket says 172.21.15.132:34780) 2024-04-23T18:35:29.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b0f7fe640 1 -- 172.21.15.132:0/3173180916 learned_addr learned my addr 172.21.15.132:0/3173180916 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b0f7fe640 1 -- 172.21.15.132:0/3173180916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc4fb80 msgr2=0x557c7bc460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b0f7fe640 1 --2- 172.21.15.132:0/3173180916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc4fb80 0x557c7bc460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b0f7fe640 1 -- 172.21.15.132:0/3173180916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc51710 msgr2=0x557c7bc49940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b0f7fe640 1 --2- 172.21.15.132:0/3173180916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc51710 0x557c7bc49940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b0f7fe640 1 -- 172.21.15.132:0/3173180916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c7bc57af0 con 0x557c7bc50d50 2024-04-23T18:35:29.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7b0f7fe640 1 --2- 172.21.15.132:0/3173180916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 0x557c7bc61540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7b04000c00 tx=0x7f7b0400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd153a1984e0a703 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7af5ffb640 1 -- 172.21.15.132:0/3173180916 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04015030 con 0x557c7bc50d50 2024-04-23T18:35:29.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7af5ffb640 1 -- 172.21.15.132:0/3173180916 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b04033020 con 0x557c7bc50d50 2024-04-23T18:35:29.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.970+0000 7f7af5ffb640 1 -- 172.21.15.132:0/3173180916 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b040118f0 con 0x557c7bc50d50 2024-04-23T18:35:29.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.971+0000 7f7b19621740 1 -- 172.21.15.132:0/3173180916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 msgr2=0x557c7bc61540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.971+0000 7f7b19621740 1 --2- 172.21.15.132:0/3173180916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 0x557c7bc61540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7b04000c00 tx=0x7f7b0400b480 comp rx=0 tx=0).stop 2024-04-23T18:35:29.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.971+0000 7f7b19621740 1 -- 172.21.15.132:0/3173180916 shutdown_connections 2024-04-23T18:35:29.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.971+0000 7f7b19621740 1 --2- 172.21.15.132:0/3173180916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc51710 0x557c7bc49940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.971+0000 7f7b19621740 1 --2- 172.21.15.132:0/3173180916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc50d50 0x557c7bc61540 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.971+0000 7f7b19621740 1 --2- 172.21.15.132:0/3173180916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc4fb80 0x557c7bc460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.971+0000 7f7b19621740 1 -- 172.21.15.132:0/3173180916 >> 172.21.15.132:0/3173180916 conn(0x557c7bc4b070 msgr2=0x557c7bc65d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.971+0000 7f7b19621740 1 -- 172.21.15.132:0/3173180916 shutdown_connections 2024-04-23T18:35:29.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.971+0000 7f7b19621740 1 -- 172.21.15.132:0/3173180916 wait complete. 2024-04-23T18:35:29.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.971+0000 7f7b19621740 1 -- start start 2024-04-23T18:35:29.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.971+0000 7f7b19621740 1 Processor -- start 2024-04-23T18:35:29.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b19621740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc4fb80 0x557c7bc5c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b19621740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc50d50 0x557c7bc5ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b19621740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 0x557c7bc5fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b19621740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x557c7bc68000 con 0x557c7bc51710 2024-04-23T18:35:29.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b19621740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557c7bc67930 con 0x557c7bc50d50 2024-04-23T18:35:29.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b19621740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557c7bc686d0 con 0x557c7bc4fb80 2024-04-23T18:35:29.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b19621740 10 client.0 _ll_register_callbacks cb 0x7f7b0400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:29.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b14dce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 0x557c7bc5fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b14dce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 0x557c7bc5fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34784/0 (socket says 172.21.15.132:34784) 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b14dce640 1 -- 172.21.15.132:0/4045169284 learned_addr learned my addr 172.21.15.132:0/4045169284 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b0f7fe640 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc50d50 0x557c7bc5ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b0ffff640 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc4fb80 0x557c7bc5c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b14dce640 1 -- 172.21.15.132:0/4045169284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc4fb80 msgr2=0x557c7bc5c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b14dce640 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc4fb80 0x557c7bc5c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b14dce640 1 -- 172.21.15.132:0/4045169284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc50d50 msgr2=0x557c7bc5ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b14dce640 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc50d50 0x557c7bc5ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b14dce640 1 -- 172.21.15.132:0/4045169284 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c7bc60440 con 0x557c7bc51710 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b14dce640 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 0x557c7bc5fe50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009c00 tx=0x7f7b08004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4495843a55e333b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7aef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7aef7fe640 1 -- 172.21.15.132:0/4045169284 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08029060 con 0x557c7bc51710 2024-04-23T18:35:29.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7aef7fe640 1 -- 172.21.15.132:0/4045169284 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7b080245f0 con 0x557c7bc51710 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b19621740 1 -- 172.21.15.132:0/4045169284 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b04033020 con 0x557c7bc51710 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7aef7fe640 1 -- 172.21.15.132:0/4045169284 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7b08014040 con 0x557c7bc51710 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b19621740 10 client.15126 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.972+0000 7f7b19621740 1 -- 172.21.15.132:0/4045169284 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c7bccccc0 con 0x557c7bc51710 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.973+0000 7f7aef7fe640 1 -- 172.21.15.132:0/4045169284 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b04033020 con 0x557c7bc51710 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.973+0000 7f7aef7fe640 1 -- 172.21.15.132:0/4045169284 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7b08023070 con 0x557c7bc51710 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.973+0000 7f7b19621740 10 client.15126 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.973+0000 7f7b19621740 10 client.15126 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.973+0000 7f7b19621740 20 client.15126 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.973+0000 7f7b19621740 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x557c7bcd18d0 0x557c7bcd05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.973+0000 7f7b19621740 4 client.15126 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.973+0000 7f7b19621740 1 -- 172.21.15.132:0/4045169284 --> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] -- command(tid 0: {"prefix": "status"}) -- 0x557c7bbbc4e0 con 0x557c7bcd18d0 2024-04-23T18:35:29.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.973+0000 7f7b0ffff640 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x557c7bcd18d0 0x557c7bcd05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:29.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.973+0000 7f7b0ffff640 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x557c7bcd18d0 0x557c7bcd05e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39801a7d2c969f0f server_cookie=83e17ac501029ec4 in_seq=0 out_seq=0 2024-04-23T18:35:29.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.973+0000 7f7aef7fe640 10 client.15126 ms_handle_connect on v2:172.21.15.148:6836/2626530887 2024-04-23T18:35:29.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7aef7fe640 1 -- 172.21.15.132:0/4045169284 <== mds.1 v2:172.21.15.148:6836/2626530887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557c7bbbc4e0 con 0x557c7bcd18d0 2024-04-23T18:35:29.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7aef7fe640 10 client.15126 handle_command_reply: tid=0 2024-04-23T18:35:29.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b19621740 1 client.15126 shutdown 2024-04-23T18:35:29.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b19621740 1 -- 172.21.15.132:0/4045169284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 msgr2=0x557c7bc5fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b19621740 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 0x557c7bc5fe50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009c00 tx=0x7f7b08004b50 comp rx=0 tx=0).stop 2024-04-23T18:35:29.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b19621740 1 -- 172.21.15.132:0/4045169284 shutdown_connections 2024-04-23T18:35:29.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b19621740 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x557c7bcd18d0 0x557c7bcd05e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b19621740 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557c7bc51710 0x557c7bc5fe50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b19621740 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7bc50d50 0x557c7bc5ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7aef7fe640 0 client.15126 ms_handle_reset on v2:172.21.15.148:6836/2626530887 2024-04-23T18:35:29.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b19621740 1 --2- 172.21.15.132:0/4045169284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557c7bc4fb80 0x557c7bc5c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b19621740 1 -- 172.21.15.132:0/4045169284 >> 172.21.15.132:0/4045169284 conn(0x557c7bc4b070 msgr2=0x557c7bc460b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b19621740 1 -- 172.21.15.132:0/4045169284 shutdown_connections 2024-04-23T18:35:29.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b19621740 1 -- 172.21.15.132:0/4045169284 wait complete. 2024-04-23T18:35:29.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b19621740 20 client.15126 trim_cache size 0 max 16384 2024-04-23T18:35:29.975 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:35:29.975 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:35:29.975 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 1, 2024-04-23T18:35:29.975 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24491, 2024-04-23T18:35:29.976 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:35:29.976 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:35:29.976 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:35:29.976 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 86.446033408999995, 2024-04-23T18:35:29.976 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 25, 2024-04-23T18:35:29.976 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 88, 2024-04-23T18:35:29.976 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:35:29.976 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 106.93368650799999 2024-04-23T18:35:29.976 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:35:29.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b16831640 1 -- 172.21.15.132:0/2343637086 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7af80a3bb0 msgr2=0x7f7af80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b16831640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7af80a3bb0 0x7f7af80a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x557c7bc50c50 tx=0x7f7b08009c30 comp rx=0 tx=0).stop 2024-04-23T18:35:29.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b16831640 1 -- 172.21.15.132:0/2343637086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b10102a10 msgr2=0x7f7b10198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:29.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.974+0000 7f7b16831640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b10102a10 0x7f7b10198270 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7b00005940 tx=0x7f7b00004430 comp rx=0 tx=0).stop 2024-04-23T18:35:29.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.975+0000 7f7b16831640 1 -- 172.21.15.132:0/2343637086 shutdown_connections 2024-04-23T18:35:29.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.975+0000 7f7b16831640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f7af80a3bb0 0x7f7af80a6070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.975+0000 7f7b16831640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b10107c70 0x7f7b1019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.975+0000 7f7b16831640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b10105340 0x7f7b101987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.975+0000 7f7b16831640 1 --2- 172.21.15.132:0/2343637086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b10102a10 0x7f7b10198270 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:29.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.975+0000 7f7b16831640 1 -- 172.21.15.132:0/2343637086 >> 172.21.15.132:0/2343637086 conn(0x7f7b100fc400 msgr2=0x7f7b10103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:29.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.975+0000 7f7b16831640 1 -- 172.21.15.132:0/2343637086 shutdown_connections 2024-04-23T18:35:29.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:29.975+0000 7f7b16831640 1 -- 172.21.15.132:0/2343637086 wait complete. 2024-04-23T18:35:29.979 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T18:35:30.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.086+0000 7f95597cd640 1 -- 172.21.15.132:0/428159900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9554105d80 msgr2=0x7f955410c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:30.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.086+0000 7f95597cd640 1 --2- 172.21.15.132:0/428159900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9554105d80 0x7f955410c610 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f954c009ca0 tx=0x7f954c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.086+0000 7f95597cd640 1 -- 172.21.15.132:0/428159900 shutdown_connections 2024-04-23T18:35:30.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.086+0000 7f95597cd640 1 --2- 172.21.15.132:0/428159900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9554105d80 0x7f955410c610 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.086+0000 7f95597cd640 1 --2- 172.21.15.132:0/428159900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95541053c0 0x7f9554105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.086+0000 7f95597cd640 1 --2- 172.21.15.132:0/428159900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95541041d0 0x7f95541045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.086+0000 7f95597cd640 1 -- 172.21.15.132:0/428159900 >> 172.21.15.132:0/428159900 conn(0x7f95540ff740 msgr2=0x7f9554101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.086+0000 7f95597cd640 1 -- 172.21.15.132:0/428159900 shutdown_connections 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.086+0000 7f95597cd640 1 -- 172.21.15.132:0/428159900 wait complete. 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.086+0000 7f95597cd640 1 Processor -- start 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f95597cd640 1 -- start start 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f95597cd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95541041d0 0x7f955419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f95597cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95541053c0 0x7f955419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f95597cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9554105d80 0x7f95541a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f95597cd640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f955410fd70 con 0x7f95541041d0 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f95597cd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f955410fbf0 con 0x7f95541053c0 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f95597cd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f955410fef0 con 0x7f9554105d80 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f9552ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95541041d0 0x7f955419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f9552ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95541041d0 0x7f955419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34788/0 (socket says 172.21.15.132:34788) 2024-04-23T18:35:30.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f9552ffd640 1 -- 172.21.15.132:0/933914635 learned_addr learned my addr 172.21.15.132:0/933914635 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:30.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f9552ffd640 1 -- 172.21.15.132:0/933914635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9554105d80 msgr2=0x7f95541a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:30.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f9552ffd640 1 --2- 172.21.15.132:0/933914635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9554105d80 0x7f95541a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f9552ffd640 1 -- 172.21.15.132:0/933914635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95541053c0 msgr2=0x7f955419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:30.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f9552ffd640 1 --2- 172.21.15.132:0/933914635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95541053c0 0x7f955419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f9552ffd640 1 -- 172.21.15.132:0/933914635 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95541a2020 con 0x7f95541041d0 2024-04-23T18:35:30.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f9552ffd640 1 --2- 172.21.15.132:0/933914635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95541041d0 0x7f955419a4a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f953c00b450 tx=0x7f953c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:30.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f954bfff640 1 -- 172.21.15.132:0/933914635 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c015020 con 0x7f95541041d0 2024-04-23T18:35:30.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f95597cd640 1 -- 172.21.15.132:0/933914635 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95541a2310 con 0x7f95541041d0 2024-04-23T18:35:30.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f954bfff640 1 -- 172.21.15.132:0/933914635 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f953c011d40 con 0x7f95541041d0 2024-04-23T18:35:30.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f954bfff640 1 -- 172.21.15.132:0/933914635 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c00f340 con 0x7f95541041d0 2024-04-23T18:35:30.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.087+0000 7f95597cd640 1 -- 172.21.15.132:0/933914635 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95541a28e0 con 0x7f95541041d0 2024-04-23T18:35:30.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f954bfff640 1 -- 172.21.15.132:0/933914635 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f953c018020 con 0x7f95541041d0 2024-04-23T18:35:30.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f955c5c4740 1 Processor -- start 2024-04-23T18:35:30.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f955c5c4740 1 -- start start 2024-04-23T18:35:30.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f955c5c4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb3b80 0x56044aeb3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f955c5c4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb4d50 0x56044aeb51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f955c5c4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 0x56044aebbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f955c5c4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56044ae18c50 con 0x56044aeb5710 2024-04-23T18:35:30.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f955c5c4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56044adb5830 con 0x56044aeb3b80 2024-04-23T18:35:30.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f955c5c4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56044ae14c10 con 0x56044aeb4d50 2024-04-23T18:35:30.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f954bfff640 1 --2- 172.21.15.132:0/933914635 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f95400a3bb0 0x7f95400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f95527fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 0x56044aebbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f95527fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 0x56044aebbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34804/0 (socket says 172.21.15.132:34804) 2024-04-23T18:35:30.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f95537fe640 1 --2- 172.21.15.132:0/933914635 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f95400a3bb0 0x7f95400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.090+0000 7f95527fc640 1 -- 172.21.15.132:0/2646184483 learned_addr learned my addr 172.21.15.132:0/2646184483 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f954bfff640 1 -- 172.21.15.132:0/933914635 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f953c0cff70 con 0x7f95541041d0 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f9552ffd640 1 --2- 172.21.15.132:0/2646184483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb4d50 0x56044aeb51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f95537fe640 1 --2- 172.21.15.132:0/2646184483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb3b80 0x56044aeb3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f95527fc640 1 -- 172.21.15.132:0/2646184483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb4d50 msgr2=0x56044aeb51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f95527fc640 1 --2- 172.21.15.132:0/2646184483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb4d50 0x56044aeb51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f95527fc640 1 -- 172.21.15.132:0/2646184483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb3b80 msgr2=0x56044aeb3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f95527fc640 1 --2- 172.21.15.132:0/2646184483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb3b80 0x56044aeb3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f95527fc640 1 -- 172.21.15.132:0/2646184483 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56044ab2b2d0 con 0x56044aeb5710 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f9552ffd640 1 --2- 172.21.15.132:0/2646184483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb4d50 0x56044aeb51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f95537fe640 1 --2- 172.21.15.132:0/933914635 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f95400a3bb0 0x7f95400a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f954c02f500 tx=0x7f954c032c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f95527fc640 1 --2- 172.21.15.132:0/2646184483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 0x56044aebbfa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f954400a9c0 tx=0x7f954400ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d98e34d0a9f966 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f95497fa640 1 -- 172.21.15.132:0/2646184483 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954400cd80 con 0x56044aeb5710 2024-04-23T18:35:30.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f95497fa640 1 -- 172.21.15.132:0/2646184483 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f954400f900 con 0x56044aeb5710 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.091+0000 7f95497fa640 1 -- 172.21.15.132:0/2646184483 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544018830 con 0x56044aeb5710 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 -- 172.21.15.132:0/2646184483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 msgr2=0x56044aebbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 --2- 172.21.15.132:0/2646184483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 0x56044aebbfa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f954400a9c0 tx=0x7f954400ae80 comp rx=0 tx=0).stop 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 -- 172.21.15.132:0/2646184483 shutdown_connections 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 --2- 172.21.15.132:0/2646184483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb4d50 0x56044aeb51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 --2- 172.21.15.132:0/2646184483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 0x56044aebbfa0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 --2- 172.21.15.132:0/2646184483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb3b80 0x56044aeb3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 -- 172.21.15.132:0/2646184483 >> 172.21.15.132:0/2646184483 conn(0x56044aeaf070 msgr2=0x56044aeaf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 -- 172.21.15.132:0/2646184483 shutdown_connections 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 -- 172.21.15.132:0/2646184483 wait complete. 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 -- start start 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 Processor -- start 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 0x56044aebe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 0x56044aebe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb5710 0x56044aec5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56044aebc720 con 0x56044aeb3b80 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56044adb3470 con 0x56044aeb4d50 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56044aebc8a0 con 0x56044aeb5710 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f955c5c4740 10 client.0 _ll_register_callbacks cb 0x56044add8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.092+0000 7f9552ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 0x56044aebe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f9552ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 0x56044aebe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34820/0 (socket says 172.21.15.132:34820) 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f9552ffd640 1 -- 172.21.15.132:0/2906622955 learned_addr learned my addr 172.21.15.132:0/2906622955 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f95537fe640 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb5710 0x56044aec5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f9552ffd640 1 -- 172.21.15.132:0/2906622955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb5710 msgr2=0x56044aec5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f9552ffd640 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb5710 0x56044aec5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f9552ffd640 1 -- 172.21.15.132:0/2906622955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 msgr2=0x56044aebe540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f9552ffd640 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 0x56044aebe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f9552ffd640 1 -- 172.21.15.132:0/2906622955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56044abf56b0 con 0x56044aeb3b80 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f95537fe640 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb5710 0x56044aec5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f9552ffd640 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 0x56044aebe000 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f953c0cdab0 tx=0x7f953c09c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c710df6cad38778b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f953a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f953a7fc640 1 -- 172.21.15.132:0/2906622955 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c0b4020 con 0x56044aeb3b80 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f953a7fc640 1 -- 172.21.15.132:0/2906622955 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f953c0a0170 con 0x56044aeb3b80 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f955c5c4740 1 -- 172.21.15.132:0/2906622955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f954400f900 con 0x56044aeb3b80 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f953a7fc640 1 -- 172.21.15.132:0/2906622955 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f953c0a05f0 con 0x56044aeb3b80 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f955c5c4740 10 client.15138 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f955c5c4740 1 -- 172.21.15.132:0/2906622955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56044aec89c0 con 0x56044aeb3b80 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f953a7fc640 1 -- 172.21.15.132:0/2906622955 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f954400f900 con 0x56044aeb3b80 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f953a7fc640 1 -- 172.21.15.132:0/2906622955 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f953c0b0ce0 con 0x56044aeb3b80 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f955c5c4740 10 client.15138 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.093+0000 7f955c5c4740 10 client.15138 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.094+0000 7f955c5c4740 20 client.15138 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:30.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.094+0000 7f955c5c4740 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56044aecb0f0 0x56044aecc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.094+0000 7f955c5c4740 4 client.15138 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.094+0000 7f955c5c4740 1 -- 172.21.15.132:0/2906622955 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56044ae26670 con 0x56044aecb0f0 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.094+0000 7f95527fc640 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56044aecb0f0 0x56044aecc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.094+0000 7f95527fc640 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56044aecb0f0 0x56044aecc560 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302cbf2e52c4c3a8 server_cookie=9796800755073fb4 in_seq=0 out_seq=0 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.094+0000 7f953a7fc640 10 client.15138 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f953a7fc640 1 -- 172.21.15.132:0/2906622955 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56044ae26670 con 0x56044aecb0f0 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f953a7fc640 10 client.15138 handle_command_reply: tid=0 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f955c5c4740 1 client.15138 shutdown 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f955c5c4740 1 -- 172.21.15.132:0/2906622955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 msgr2=0x56044aebe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f955c5c4740 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 0x56044aebe000 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f953c0cdab0 tx=0x7f953c09c940 comp rx=0 tx=0).stop 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f955c5c4740 1 -- 172.21.15.132:0/2906622955 shutdown_connections 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f955c5c4740 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56044aecb0f0 0x56044aecc560 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f955c5c4740 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb5710 0x56044aec5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f953a7fc640 0 client.15138 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:35:30.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f955c5c4740 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 0x56044aebe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f955c5c4740 1 --2- 172.21.15.132:0/2906622955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 0x56044aebe000 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f955c5c4740 1 -- 172.21.15.132:0/2906622955 >> 172.21.15.132:0/2906622955 conn(0x56044aeaf070 msgr2=0x56044aeae1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:30.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f955c5c4740 1 -- 172.21.15.132:0/2906622955 shutdown_connections 2024-04-23T18:35:30.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f955c5c4740 1 -- 172.21.15.132:0/2906622955 wait complete. 2024-04-23T18:35:30.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.095+0000 7f955c5c4740 20 client.15138 trim_cache size 0 max 16384 2024-04-23T18:35:30.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.113+0000 7f955c5c4740 1 Processor -- start 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.113+0000 7f955c5c4740 1 -- start start 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.113+0000 7f955c5c4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb3b80 0x56044aebbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.113+0000 7f955c5c4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 0x56044aec5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.113+0000 7f955c5c4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 0x56044aebdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.113+0000 7f955c5c4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56044aeca8a0 con 0x56044aeb5710 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.113+0000 7f955c5c4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56044aecaf70 con 0x56044aeb4d50 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.113+0000 7f955c5c4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56044aecb640 con 0x56044aeb3b80 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.113+0000 7f95537fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 0x56044aebdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.113+0000 7f95537fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 0x56044aebdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34826/0 (socket says 172.21.15.132:34826) 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.113+0000 7f95537fe640 1 -- 172.21.15.132:0/1790529035 learned_addr learned my addr 172.21.15.132:0/1790529035 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.113+0000 7f95527fc640 1 --2- 172.21.15.132:0/1790529035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 0x56044aec5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.114+0000 7f95537fe640 1 -- 172.21.15.132:0/1790529035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb3b80 msgr2=0x56044aebbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.114+0000 7f95537fe640 1 --2- 172.21.15.132:0/1790529035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb3b80 0x56044aebbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.114+0000 7f95537fe640 1 -- 172.21.15.132:0/1790529035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 msgr2=0x56044aec5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.114+0000 7f95537fe640 1 --2- 172.21.15.132:0/1790529035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 0x56044aec5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.114+0000 7f95537fe640 1 -- 172.21.15.132:0/1790529035 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56044aebf130 con 0x56044aeb5710 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.114+0000 7f95537fe640 1 --2- 172.21.15.132:0/1790529035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 0x56044aebdc10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f954c009ca0 tx=0x7f954c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d67f37c91cd267 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.114+0000 7f9548ff9640 1 -- 172.21.15.132:0/1790529035 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c02fdb0 con 0x56044aeb5710 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.114+0000 7f9548ff9640 1 -- 172.21.15.132:0/1790529035 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f954c036040 con 0x56044aeb5710 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.114+0000 7f9548ff9640 1 -- 172.21.15.132:0/1790529035 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c03d710 con 0x56044aeb5710 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.114+0000 7f955c5c4740 1 -- 172.21.15.132:0/1790529035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 msgr2=0x56044aebdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.114+0000 7f955c5c4740 1 --2- 172.21.15.132:0/1790529035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 0x56044aebdc10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f954c009ca0 tx=0x7f954c0093f0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 -- 172.21.15.132:0/1790529035 shutdown_connections 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 --2- 172.21.15.132:0/1790529035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb5710 0x56044aebdc10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 --2- 172.21.15.132:0/1790529035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 0x56044aec5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 --2- 172.21.15.132:0/1790529035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb3b80 0x56044aebbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 -- 172.21.15.132:0/1790529035 >> 172.21.15.132:0/1790529035 conn(0x56044aeaf070 msgr2=0x56044aeaa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:30.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 -- 172.21.15.132:0/1790529035 shutdown_connections 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 -- 172.21.15.132:0/1790529035 wait complete. 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 -- start start 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 Processor -- start 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 0x56044aec0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 0x56044aec12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb5710 0x56044aec42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56044aecc3e0 con 0x56044aeb3b80 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56044aecbd10 con 0x56044aeb4d50 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56044aeccab0 con 0x56044aeb5710 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f955c5c4740 10 client.0 _ll_register_callbacks cb 0x7f953c09bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f9552ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 0x56044aec0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f9552ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 0x56044aec0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34834/0 (socket says 172.21.15.132:34834) 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f9552ffd640 1 -- 172.21.15.132:0/266118236 learned_addr learned my addr 172.21.15.132:0/266118236 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:30.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f95527fc640 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 0x56044aec12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f95537fe640 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb5710 0x56044aec42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f9552ffd640 1 -- 172.21.15.132:0/266118236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb5710 msgr2=0x56044aec42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f9552ffd640 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb5710 0x56044aec42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f9552ffd640 1 -- 172.21.15.132:0/266118236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 msgr2=0x56044aec12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f9552ffd640 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 0x56044aec12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f9552ffd640 1 -- 172.21.15.132:0/266118236 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56044aec48c0 con 0x56044aeb3b80 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f95527fc640 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 0x56044aec12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.115+0000 7f9552ffd640 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 0x56044aec0d70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f953c00be10 tx=0x7f953c0a4c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a343b9f359bdcde6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f953a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f953a7fc640 1 -- 172.21.15.132:0/266118236 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c0b4070 con 0x56044aeb3b80 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f953a7fc640 1 -- 172.21.15.132:0/266118236 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f953c0a0170 con 0x56044aeb3b80 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f955c5c4740 1 -- 172.21.15.132:0/266118236 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f954c036040 con 0x56044aeb3b80 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f953a7fc640 1 -- 172.21.15.132:0/266118236 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f953c0a0370 con 0x56044aeb3b80 2024-04-23T18:35:30.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f955c5c4740 10 client.15144 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f955c5c4740 1 -- 172.21.15.132:0/266118236 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56044af31160 con 0x56044aeb3b80 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f953a7fc640 1 -- 172.21.15.132:0/266118236 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f954c036040 con 0x56044aeb3b80 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f953a7fc640 1 -- 172.21.15.132:0/266118236 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f953c0b0ce0 con 0x56044aeb3b80 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f955c5c4740 10 client.15144 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f955c5c4740 10 client.15144 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f955c5c4740 20 client.15144 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f955c5c4740 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56044aecb0f0 0x56044af346e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f955c5c4740 4 client.15144 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f955c5c4740 1 -- 172.21.15.132:0/266118236 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "status"}) -- 0x56044ae204e0 con 0x56044aecb0f0 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.116+0000 7f95527fc640 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56044aecb0f0 0x56044af346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f953a7fc640 10 client.15144 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f95527fc640 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56044aecb0f0 0x56044af346e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45bd29abf5025dda server_cookie=b701533513f8f53a in_seq=0 out_seq=0 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f953a7fc640 1 -- 172.21.15.132:0/266118236 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56044ae204e0 con 0x56044aecb0f0 2024-04-23T18:35:30.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f953a7fc640 10 client.15144 handle_command_reply: tid=0 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f955c5c4740 1 client.15144 shutdown 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f955c5c4740 1 -- 172.21.15.132:0/266118236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 msgr2=0x56044aec0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f955c5c4740 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 0x56044aec0d70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f953c00be10 tx=0x7f953c0a4c50 comp rx=0 tx=0).stop 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f955c5c4740 1 -- 172.21.15.132:0/266118236 shutdown_connections 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f955c5c4740 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56044aecb0f0 0x56044af346e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f955c5c4740 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56044aeb5710 0x56044aec42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f953a7fc640 0 client.15144 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f955c5c4740 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56044aeb4d50 0x56044aec12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f955c5c4740 1 --2- 172.21.15.132:0/266118236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044aeb3b80 0x56044aec0d70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f955c5c4740 1 -- 172.21.15.132:0/266118236 >> 172.21.15.132:0/266118236 conn(0x56044aeaf070 msgr2=0x56044aeb87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f955c5c4740 1 -- 172.21.15.132:0/266118236 shutdown_connections 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f955c5c4740 1 -- 172.21.15.132:0/266118236 wait complete. 2024-04-23T18:35:30.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.117+0000 7f955c5c4740 20 client.15144 trim_cache size 0 max 16384 2024-04-23T18:35:30.117 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:35:30.117 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:35:30.117 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 0, 2024-04-23T18:35:30.117 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24515, 2024-04-23T18:35:30.117 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:35:30.117 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:35:30.117 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:35:30.117 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 88.601554902000004, 2024-04-23T18:35:30.117 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 25, 2024-04-23T18:35:30.117 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 88, 2024-04-23T18:35:30.117 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:35:30.118 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 95.596722313000001 2024-04-23T18:35:30.118 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:35:30.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.118+0000 7f95597cd640 1 -- 172.21.15.132:0/933914635 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f95400a3bb0 msgr2=0x7f95400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:30.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.118+0000 7f95597cd640 1 --2- 172.21.15.132:0/933914635 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f95400a3bb0 0x7f95400a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f954c02f500 tx=0x7f954c032c00 comp rx=0 tx=0).stop 2024-04-23T18:35:30.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.118+0000 7f95597cd640 1 -- 172.21.15.132:0/933914635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95541041d0 msgr2=0x7f955419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:30.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.118+0000 7f95597cd640 1 --2- 172.21.15.132:0/933914635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95541041d0 0x7f955419a4a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f953c00b450 tx=0x7f953c00b910 comp rx=0 tx=0).stop 2024-04-23T18:35:30.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.118+0000 7f95597cd640 1 -- 172.21.15.132:0/933914635 shutdown_connections 2024-04-23T18:35:30.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.118+0000 7f95597cd640 1 --2- 172.21.15.132:0/933914635 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f95400a3bb0 0x7f95400a6070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.118+0000 7f95597cd640 1 --2- 172.21.15.132:0/933914635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9554105d80 0x7f95541a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.118+0000 7f95597cd640 1 --2- 172.21.15.132:0/933914635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95541053c0 0x7f955419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.118+0000 7f95597cd640 1 --2- 172.21.15.132:0/933914635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95541041d0 0x7f955419a4a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:30.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.118+0000 7f95597cd640 1 -- 172.21.15.132:0/933914635 >> 172.21.15.132:0/933914635 conn(0x7f95540ff740 msgr2=0x7f95541086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:30.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.118+0000 7f95597cd640 1 -- 172.21.15.132:0/933914635 shutdown_connections 2024-04-23T18:35:30.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:30.118+0000 7f95597cd640 1 -- 172.21.15.132:0/933914635 wait complete. 2024-04-23T18:35:30.121 INFO:tasks.cephadm:Setup complete, yielding 2024-04-23T18:35:30.121 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T18:35:30.134 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-04-23T18:35:30.134 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-23T18:35:30.374 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:30.496 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:30 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3972549329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:30.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:30 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3972549329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:30.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:30 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3972549329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:31.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:31 smithi148 ceph-mon[32311]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:35:31.520 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:31 smithi132 ceph-mon[29178]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:35:31.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:31 smithi157 ceph-mon[32580]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:35:32.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.089+0000 7f05c16c1700 1 -- 172.21.15.132:0/2473902117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05bc1026b0 msgr2=0x7f05bc102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:32.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.089+0000 7f05c16c1700 1 --2- 172.21.15.132:0/2473902117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05bc1026b0 0x7f05bc102b30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009990 tx=0x7f05b4009ca0 comp rx=0 tx=0).stop 2024-04-23T18:35:32.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.089+0000 7f05c16c1700 1 -- 172.21.15.132:0/2473902117 shutdown_connections 2024-04-23T18:35:32.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.089+0000 7f05c16c1700 1 --2- 172.21.15.132:0/2473902117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05bc1026b0 0x7f05bc102b30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:32.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.089+0000 7f05c16c1700 1 --2- 172.21.15.132:0/2473902117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05bc101390 0x7f05bc1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.089+0000 7f05c16c1700 1 --2- 172.21.15.132:0/2473902117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05bc101cf0 0x7f05bc102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.089+0000 7f05c16c1700 1 -- 172.21.15.132:0/2473902117 >> 172.21.15.132:0/2473902117 conn(0x7f05bc0fcf30 msgr2=0x7f05bc0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.089+0000 7f05c16c1700 1 -- 172.21.15.132:0/2473902117 shutdown_connections 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.089+0000 7f05c16c1700 1 -- 172.21.15.132:0/2473902117 wait complete. 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.089+0000 7f05c16c1700 1 Processor -- start 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05c16c1700 1 -- start start 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05c16c1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05bc101390 0x7f05bc19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05c16c1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05bc101cf0 0x7f05bc19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05c16c1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05bc1026b0 0x7f05bc1a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05c16c1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc110760 con 0x7f05bc101390 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05c16c1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc1105e0 con 0x7f05bc101cf0 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05c16c1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc1108e0 con 0x7f05bc1026b0 2024-04-23T18:35:32.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05baffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05bc101390 0x7f05bc19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05baffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05bc101390 0x7f05bc19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34864/0 (socket says 172.21.15.132:34864) 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05baffd700 1 -- 172.21.15.132:0/3995779779 learned_addr learned my addr 172.21.15.132:0/3995779779 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05ba7fc700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05bc101cf0 0x7f05bc19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05bb7fe700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05bc1026b0 0x7f05bc1a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05baffd700 1 -- 172.21.15.132:0/3995779779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05bc1026b0 msgr2=0x7f05bc1a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05baffd700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05bc1026b0 0x7f05bc1a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05baffd700 1 -- 172.21.15.132:0/3995779779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05bc101cf0 msgr2=0x7f05bc19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05baffd700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05bc101cf0 0x7f05bc19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05baffd700 1 -- 172.21.15.132:0/3995779779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05bc1a2a00 con 0x7f05bc101390 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05bb7fe700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05bc1026b0 0x7f05bc1a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05baffd700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05bc101390 0x7f05bc19ada0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00ba80 tx=0x7f05ac00be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05b3fff700 1 -- 172.21.15.132:0/3995779779 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac019070 con 0x7f05bc101390 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.090+0000 7f05c16c1700 1 -- 172.21.15.132:0/3995779779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05bc1a2cf0 con 0x7f05bc101390 2024-04-23T18:35:32.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.091+0000 7f05b3fff700 1 -- 172.21.15.132:0/3995779779 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05ac014440 con 0x7f05bc101390 2024-04-23T18:35:32.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.091+0000 7f05b3fff700 1 -- 172.21.15.132:0/3995779779 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0075d0 con 0x7f05bc101390 2024-04-23T18:35:32.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.091+0000 7f05c16c1700 1 -- 172.21.15.132:0/3995779779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05bc1a3200 con 0x7f05bc101390 2024-04-23T18:35:32.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.092+0000 7f05c16c1700 1 -- 172.21.15.132:0/3995779779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05bc02df80 con 0x7f05bc101390 2024-04-23T18:35:32.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.093+0000 7f05b3fff700 1 -- 172.21.15.132:0/3995779779 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f05ac01c070 con 0x7f05bc101390 2024-04-23T18:35:32.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.093+0000 7f05b3fff700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f05a80a3c40 0x7f05a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:32.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.093+0000 7f05ba7fc700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f05a80a3c40 0x7f05a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:32.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.093+0000 7f05b3fff700 1 -- 172.21.15.132:0/3995779779 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f05ac0cf8a0 con 0x7f05bc101390 2024-04-23T18:35:32.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.094+0000 7f05ba7fc700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f05a80a3c40 0x7f05a80a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f05a4005fd0 tx=0x7f05a400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:32.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.098+0000 7f05b3fff700 1 -- 172.21.15.132:0/3995779779 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05ac09c270 con 0x7f05bc101390 2024-04-23T18:35:32.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.321+0000 7f05c16c1700 1 -- 172.21.15.132:0/3995779779 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f05bc062e80 con 0x7f05a80a3c40 2024-04-23T18:35:32.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.322+0000 7f05b3fff700 1 -- 172.21.15.132:0/3995779779 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f05bc062e80 con 0x7f05a80a3c40 2024-04-23T18:35:32.314 INFO:teuthology.orchestra.run.smithi132.stdout:Backend: cephadm 2024-04-23T18:35:32.314 INFO:teuthology.orchestra.run.smithi132.stdout:Available: Yes 2024-04-23T18:35:32.314 INFO:teuthology.orchestra.run.smithi132.stdout:Paused: No 2024-04-23T18:35:32.316 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.324+0000 7f05c16c1700 1 -- 172.21.15.132:0/3995779779 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f05a80a3c40 msgr2=0x7f05a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:32.316 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.324+0000 7f05c16c1700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f05a80a3c40 0x7f05a80a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f05a4005fd0 tx=0x7f05a400a560 comp rx=0 tx=0).stop 2024-04-23T18:35:32.316 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.324+0000 7f05c16c1700 1 -- 172.21.15.132:0/3995779779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05bc101390 msgr2=0x7f05bc19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:32.316 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.324+0000 7f05c16c1700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05bc101390 0x7f05bc19ada0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00ba80 tx=0x7f05ac00be40 comp rx=0 tx=0).stop 2024-04-23T18:35:32.316 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.324+0000 7f05c16c1700 1 -- 172.21.15.132:0/3995779779 shutdown_connections 2024-04-23T18:35:32.316 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.324+0000 7f05c16c1700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05bc1026b0 0x7f05bc1a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:32.317 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.324+0000 7f05c16c1700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f05a80a3c40 0x7f05a80a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:32.317 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.324+0000 7f05c16c1700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05bc101390 0x7f05bc19ada0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:32.317 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.324+0000 7f05c16c1700 1 --2- 172.21.15.132:0/3995779779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05bc101cf0 0x7f05bc19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:32.317 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.324+0000 7f05c16c1700 1 -- 172.21.15.132:0/3995779779 >> 172.21.15.132:0/3995779779 conn(0x7f05bc0fcf30 msgr2=0x7f05bc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:32.317 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.324+0000 7f05c16c1700 1 -- 172.21.15.132:0/3995779779 shutdown_connections 2024-04-23T18:35:32.317 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:32.324+0000 7f05c16c1700 1 -- 172.21.15.132:0/3995779779 wait complete. 2024-04-23T18:35:33.091 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-23T18:35:33.332 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:33.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:33 smithi132 ceph-mon[29178]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-23T18:35:33.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:33 smithi148 ceph-mon[32311]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-23T18:35:33.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:33 smithi157 ceph-mon[32580]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-23T18:35:34.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:34 smithi148 ceph-mon[32311]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:34.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:34 smithi157 ceph-mon[32580]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:34.562 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:34 smithi132 ceph-mon[29178]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:35.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.126+0000 7f8c36e10700 1 -- 172.21.15.132:0/4131891449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c30103540 msgr2=0x7f8c30109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.126+0000 7f8c36e10700 1 --2- 172.21.15.132:0/4131891449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c30103540 0x7f8c30109df0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800b0d0 tx=0x7f8c2800b3e0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.126+0000 7f8c36e10700 1 -- 172.21.15.132:0/4131891449 shutdown_connections 2024-04-23T18:35:35.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.126+0000 7f8c36e10700 1 --2- 172.21.15.132:0/4131891449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c30103540 0x7f8c30109df0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.126+0000 7f8c36e10700 1 --2- 172.21.15.132:0/4131891449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c301019a0 0x7f8c30101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.126+0000 7f8c36e10700 1 --2- 172.21.15.132:0/4131891449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c30102ba0 0x7f8c30103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.126+0000 7f8c36e10700 1 -- 172.21.15.132:0/4131891449 >> 172.21.15.132:0/4131891449 conn(0x7f8c300fcf40 msgr2=0x7f8c300ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.126+0000 7f8c36e10700 1 -- 172.21.15.132:0/4131891449 shutdown_connections 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.126+0000 7f8c36e10700 1 -- 172.21.15.132:0/4131891449 wait complete. 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c36e10700 1 Processor -- start 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c36e10700 1 -- start start 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c36e10700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c301019a0 0x7f8c30197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c36e10700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c30102ba0 0x7f8c30197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c36e10700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c30103540 0x7f8c3019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c36e10700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3010d440 con 0x7f8c30102ba0 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c36e10700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3010d2c0 con 0x7f8c30103540 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c36e10700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3010d5c0 con 0x7f8c301019a0 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c2ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c30102ba0 0x7f8c30197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c2ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c30102ba0 0x7f8c30197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34886/0 (socket says 172.21.15.132:34886) 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c2ffff700 1 -- 172.21.15.132:0/2711129115 learned_addr learned my addr 172.21.15.132:0/2711129115 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c34bac700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c301019a0 0x7f8c30197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c353ad700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c30103540 0x7f8c3019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c2ffff700 1 -- 172.21.15.132:0/2711129115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c301019a0 msgr2=0x7f8c30197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c2ffff700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c301019a0 0x7f8c30197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c2ffff700 1 -- 172.21.15.132:0/2711129115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c30103540 msgr2=0x7f8c3019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c2ffff700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c30103540 0x7f8c3019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c2ffff700 1 -- 172.21.15.132:0/2711129115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3019f6b0 con 0x7f8c30102ba0 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c353ad700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c30103540 0x7f8c3019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c2ffff700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c30102ba0 0x7f8c30197f90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00c9a0 tx=0x7f8c1c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c2dffb700 1 -- 172.21.15.132:0/2711129115 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c007a60 con 0x7f8c30102ba0 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c2dffb700 1 -- 172.21.15.132:0/2711129115 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c1c004ce0 con 0x7f8c30102ba0 2024-04-23T18:35:35.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c36e10700 1 -- 172.21.15.132:0/2711129115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3019f9a0 con 0x7f8c30102ba0 2024-04-23T18:35:35.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.127+0000 7f8c2dffb700 1 -- 172.21.15.132:0/2711129115 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c0055c0 con 0x7f8c30102ba0 2024-04-23T18:35:35.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.128+0000 7f8c36e10700 1 -- 172.21.15.132:0/2711129115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c3019fe90 con 0x7f8c30102ba0 2024-04-23T18:35:35.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.129+0000 7f8c36e10700 1 -- 172.21.15.132:0/2711129115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c30069730 con 0x7f8c30102ba0 2024-04-23T18:35:35.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.130+0000 7f8c2dffb700 1 -- 172.21.15.132:0/2711129115 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f8c1c007c00 con 0x7f8c30102ba0 2024-04-23T18:35:35.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.130+0000 7f8c2dffb700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8c200a3c40 0x7f8c200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.131+0000 7f8c34bac700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8c200a3c40 0x7f8c200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.131+0000 7f8c2dffb700 1 -- 172.21.15.132:0/2711129115 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f8c1c0d0510 con 0x7f8c30102ba0 2024-04-23T18:35:35.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.131+0000 7f8c34bac700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8c200a3c40 0x7f8c200a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8c30102a00 tx=0x7f8c24005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:35.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.135+0000 7f8c2dffb700 1 -- 172.21.15.132:0/2711129115 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c1c09d000 con 0x7f8c30102ba0 2024-04-23T18:35:35.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.345+0000 7f8c36e10700 1 -- 172.21.15.132:0/2711129115 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8c30062e80 con 0x7f8c200a3c40 2024-04-23T18:35:35.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.359+0000 7f8c2dffb700 1 -- 172.21.15.132:0/2711129115 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f8c30062e80 con 0x7f8c200a3c40 2024-04-23T18:35:35.351 INFO:teuthology.orchestra.run.smithi132.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T18:35:35.351 INFO:teuthology.orchestra.run.smithi132.stdout:mds.a smithi132 running (104s) 16s ago 103s 19.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d826cab44e48 2024-04-23T18:35:35.351 INFO:teuthology.orchestra.run.smithi132.stdout:mds.b smithi148 running (101s) 34s ago 100s 26.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 9219f4346f4c 2024-04-23T18:35:35.351 INFO:teuthology.orchestra.run.smithi132.stdout:mds.c smithi157 running (115s) 34s ago 115s 16.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 e805f720b6ee 2024-04-23T18:35:35.351 INFO:teuthology.orchestra.run.smithi132.stdout:mds.d smithi132 running (2m) 16s ago 2m 16.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a61f079d5fe7 2024-04-23T18:35:35.351 INFO:teuthology.orchestra.run.smithi132.stdout:mds.e smithi148 running (2m) 34s ago 2m 18.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2c46f700e04f 2024-04-23T18:35:35.351 INFO:teuthology.orchestra.run.smithi132.stdout:mds.f smithi157 running (118s) 34s ago 117s 19.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 fdfc1d81fa7f 2024-04-23T18:35:35.351 INFO:teuthology.orchestra.run.smithi132.stdout:mds.g smithi132 running (107s) 16s ago 106s 18.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 fcb9452407ea 2024-04-23T18:35:35.351 INFO:teuthology.orchestra.run.smithi132.stdout:mds.h smithi148 running (112s) 34s ago 112s 20.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a50d964884fa 2024-04-23T18:35:35.352 INFO:teuthology.orchestra.run.smithi132.stdout:mds.i smithi157 running (109s) 34s ago 109s 16.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 e3360bcde070 2024-04-23T18:35:35.352 INFO:teuthology.orchestra.run.smithi132.stdout:mds.j smithi132 running (2m) 16s ago 2m 16.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 59874b8d89d2 2024-04-23T18:35:35.352 INFO:teuthology.orchestra.run.smithi132.stdout:mds.k smithi148 running (2m) 34s ago 2m 18.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1c7c47ae3cda 2024-04-23T18:35:35.352 INFO:teuthology.orchestra.run.smithi132.stdout:mds.l smithi157 running (2m) 34s ago 2m 17.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d2b89506d0e6 2024-04-23T18:35:35.352 INFO:teuthology.orchestra.run.smithi132.stdout:mgr.x smithi132 *:9283,8765 running (9m) 16s ago 9m 514M - 19.0.0-3156-gd69f13d8 da4b004c09e8 390d7bfdf89b 2024-04-23T18:35:35.352 INFO:teuthology.orchestra.run.smithi132.stdout:mgr.y smithi148 *:8443,8765 running (7m) 34s ago 7m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 203519ea92c8 2024-04-23T18:35:35.352 INFO:teuthology.orchestra.run.smithi132.stdout:mgr.z smithi157 *:8443,8765 running (7m) 34s ago 7m 453M - 19.0.0-3156-gd69f13d8 da4b004c09e8 7f74d05ed853 2024-04-23T18:35:35.352 INFO:teuthology.orchestra.run.smithi132.stdout:mon.a smithi132 running (9m) 16s ago 9m 50.0M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 2bb1a8944af2 2024-04-23T18:35:35.352 INFO:teuthology.orchestra.run.smithi132.stdout:mon.b smithi148 running (7m) 34s ago 7m 42.4M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 dff4ea7f056d 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:mon.c smithi157 running (7m) 34s ago 7m 39.5M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 ff4c6dfe43a8 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:osd.0 smithi132 running (7m) 16s ago 7m 46.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 7226ebbf5a58 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:osd.1 smithi148 running (6m) 34s ago 6m 45.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 6d28072d31d3 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:osd.2 smithi157 running (6m) 34s ago 6m 59.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 1eb04c4bad56 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:osd.3 smithi132 running (5m) 16s ago 5m 43.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 36ade6962579 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:osd.4 smithi148 running (5m) 34s ago 5m 42.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 cf90c0fe091e 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:osd.5 smithi157 running (5m) 34s ago 5m 43.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 e3d0602e146f 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:osd.6 smithi132 running (4m) 16s ago 4m 53.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 3c3d6b3ad09f 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:osd.7 smithi148 running (4m) 34s ago 4m 52.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 6d3988c0c592 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:osd.8 smithi157 running (3m) 34s ago 3m 42.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b96f5af7c11a 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:osd.9 smithi132 running (3m) 16s ago 3m 44.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 1c9bfb1d7c69 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:osd.10 smithi148 running (2m) 34s ago 2m 43.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b767055a7ec4 2024-04-23T18:35:35.353 INFO:teuthology.orchestra.run.smithi132.stdout:osd.11 smithi157 running (2m) 34s ago 2m 41.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 fc9b31f71359 2024-04-23T18:35:35.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.361+0000 7f8c36e10700 1 -- 172.21.15.132:0/2711129115 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8c200a3c40 msgr2=0x7f8c200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.361+0000 7f8c36e10700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8c200a3c40 0x7f8c200a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8c30102a00 tx=0x7f8c24005d70 comp rx=0 tx=0).stop 2024-04-23T18:35:35.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.361+0000 7f8c36e10700 1 -- 172.21.15.132:0/2711129115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c30102ba0 msgr2=0x7f8c30197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.361+0000 7f8c36e10700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c30102ba0 0x7f8c30197f90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00c9a0 tx=0x7f8c1c00cd60 comp rx=0 tx=0).stop 2024-04-23T18:35:35.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.361+0000 7f8c36e10700 1 -- 172.21.15.132:0/2711129115 shutdown_connections 2024-04-23T18:35:35.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.361+0000 7f8c36e10700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c30103540 0x7f8c3019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.361+0000 7f8c36e10700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c301019a0 0x7f8c30197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.361+0000 7f8c36e10700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f8c200a3c40 0x7f8c200a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.361+0000 7f8c36e10700 1 --2- 172.21.15.132:0/2711129115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c30102ba0 0x7f8c30197f90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.361+0000 7f8c36e10700 1 -- 172.21.15.132:0/2711129115 >> 172.21.15.132:0/2711129115 conn(0x7f8c300fcf40 msgr2=0x7f8c30105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:35.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.361+0000 7f8c36e10700 1 -- 172.21.15.132:0/2711129115 shutdown_connections 2024-04-23T18:35:35.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:35.361+0000 7f8c36e10700 1 -- 172.21.15.132:0/2711129115 wait complete. 2024-04-23T18:35:35.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:35 smithi148 ceph-mon[32311]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T18:35:35.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:35 smithi157 ceph-mon[32580]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T18:35:35.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:35 smithi132 ceph-mon[29178]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T18:35:36.140 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-23T18:35:36.382 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:37.514 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:37 smithi132 ceph-mon[29178]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:37.514 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:37 smithi132 ceph-mon[29178]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:35:37.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:37 smithi148 ceph-mon[32311]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:37.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:37 smithi148 ceph-mon[32311]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:35:37.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:37 smithi157 ceph-mon[32580]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:37.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:37 smithi157 ceph-mon[32580]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:35:38.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.076+0000 7f5e82409700 1 -- 172.21.15.132:0/4077297256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e7c102ba0 msgr2=0x7f5e7c103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:38.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.076+0000 7f5e82409700 1 --2- 172.21.15.132:0/4077297256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e7c102ba0 0x7f5e7c103000 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5e700099c0 tx=0x7f5e70009cd0 comp rx=0 tx=0).stop 2024-04-23T18:35:38.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.076+0000 7f5e82409700 1 -- 172.21.15.132:0/4077297256 shutdown_connections 2024-04-23T18:35:38.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.076+0000 7f5e82409700 1 --2- 172.21.15.132:0/4077297256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e7c103540 0x7f5e7c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:38.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.076+0000 7f5e82409700 1 --2- 172.21.15.132:0/4077297256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7c1019a0 0x7f5e7c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:38.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.076+0000 7f5e82409700 1 --2- 172.21.15.132:0/4077297256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e7c102ba0 0x7f5e7c103000 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:38.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.076+0000 7f5e82409700 1 -- 172.21.15.132:0/4077297256 >> 172.21.15.132:0/4077297256 conn(0x7f5e7c0fcf40 msgr2=0x7f5e7c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:38.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.076+0000 7f5e82409700 1 -- 172.21.15.132:0/4077297256 shutdown_connections 2024-04-23T18:35:38.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.076+0000 7f5e82409700 1 -- 172.21.15.132:0/4077297256 wait complete. 2024-04-23T18:35:38.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.077+0000 7f5e82409700 1 Processor -- start 2024-04-23T18:35:38.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.077+0000 7f5e82409700 1 -- start start 2024-04-23T18:35:38.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.077+0000 7f5e82409700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e7c1019a0 0x7f5e7c06b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:38.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.077+0000 7f5e82409700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e7c102ba0 0x7f5e7c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:38.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.077+0000 7f5e82409700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7c103540 0x7f5e7c19ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:38.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.077+0000 7f5e82409700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c10d440 con 0x7f5e7c1019a0 2024-04-23T18:35:38.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.077+0000 7f5e82409700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c10d2c0 con 0x7f5e7c103540 2024-04-23T18:35:38.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.077+0000 7f5e82409700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c10d5c0 con 0x7f5e7c102ba0 2024-04-23T18:35:38.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.077+0000 7f5e7bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e7c1019a0 0x7f5e7c06b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:38.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.077+0000 7f5e7bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e7c1019a0 0x7f5e7c06b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34896/0 (socket says 172.21.15.132:34896) 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.077+0000 7f5e7bfff700 1 -- 172.21.15.132:0/3827639613 learned_addr learned my addr 172.21.15.132:0/3827639613 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.077+0000 7f5e7b7fe700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e7c102ba0 0x7f5e7c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.078+0000 7f5e809a6700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7c103540 0x7f5e7c19ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.078+0000 7f5e7bfff700 1 -- 172.21.15.132:0/3827639613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e7c102ba0 msgr2=0x7f5e7c19d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.078+0000 7f5e7bfff700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e7c102ba0 0x7f5e7c19d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.078+0000 7f5e7bfff700 1 -- 172.21.15.132:0/3827639613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7c103540 msgr2=0x7f5e7c19ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.078+0000 7f5e7bfff700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7c103540 0x7f5e7c19ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.078+0000 7f5e7bfff700 1 -- 172.21.15.132:0/3827639613 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c1a0890 con 0x7f5e7c1019a0 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.078+0000 7f5e7bfff700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e7c1019a0 0x7f5e7c06b2a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00e970 tx=0x7f5e6c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.078+0000 7f5e797fa700 1 -- 172.21.15.132:0/3827639613 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00cbd0 con 0x7f5e7c1019a0 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.078+0000 7f5e82409700 1 -- 172.21.15.132:0/3827639613 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7c1a0b80 con 0x7f5e7c1019a0 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.078+0000 7f5e797fa700 1 -- 172.21.15.132:0/3827639613 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c0044d0 con 0x7f5e7c1019a0 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.078+0000 7f5e797fa700 1 -- 172.21.15.132:0/3827639613 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c003df0 con 0x7f5e7c1019a0 2024-04-23T18:35:38.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.078+0000 7f5e82409700 1 -- 172.21.15.132:0/3827639613 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7c1a1110 con 0x7f5e7c1019a0 2024-04-23T18:35:38.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.079+0000 7f5e82409700 1 -- 172.21.15.132:0/3827639613 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7c069730 con 0x7f5e7c1019a0 2024-04-23T18:35:38.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.080+0000 7f5e797fa700 1 -- 172.21.15.132:0/3827639613 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f5e6c003f90 con 0x7f5e7c1019a0 2024-04-23T18:35:38.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.081+0000 7f5e797fa700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5e640a3cc0 0x7f5e640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:38.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.081+0000 7f5e7b7fe700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5e640a3cc0 0x7f5e640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:38.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.081+0000 7f5e797fa700 1 -- 172.21.15.132:0/3827639613 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5e6c0d0740 con 0x7f5e7c1019a0 2024-04-23T18:35:38.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.082+0000 7f5e7b7fe700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5e640a3cc0 0x7f5e640a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c06b800 tx=0x7f5e70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:38.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.085+0000 7f5e797fa700 1 -- 172.21.15.132:0/3827639613 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e6c09d140 con 0x7f5e7c1019a0 2024-04-23T18:35:38.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.294+0000 7f5e82409700 1 -- 172.21.15.132:0/3827639613 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f5e7c062e80 con 0x7f5e640a3cc0 2024-04-23T18:35:38.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.299+0000 7f5e797fa700 1 -- 172.21.15.132:0/3827639613 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f5e7c062e80 con 0x7f5e640a3cc0 2024-04-23T18:35:38.292 INFO:teuthology.orchestra.run.smithi132.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-23T18:35:38.292 INFO:teuthology.orchestra.run.smithi132.stdout:mds.all 12/12 37s ago 2m smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-04-23T18:35:38.292 INFO:teuthology.orchestra.run.smithi132.stdout:mgr 3/3 37s ago 7m smithi132=x;smithi148=y;smithi157=z;count:3 2024-04-23T18:35:38.292 INFO:teuthology.orchestra.run.smithi132.stdout:mon 3/3 37s ago 8m smithi132:172.21.15.132=a;smithi148:172.21.15.148=b;smithi157:172.21.15.157=c;count:3 2024-04-23T18:35:38.292 INFO:teuthology.orchestra.run.smithi132.stdout:osd 12 37s ago - 2024-04-23T18:35:38.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.302+0000 7f5e82409700 1 -- 172.21.15.132:0/3827639613 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5e640a3cc0 msgr2=0x7f5e640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:38.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.302+0000 7f5e82409700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5e640a3cc0 0x7f5e640a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c06b800 tx=0x7f5e70005fb0 comp rx=0 tx=0).stop 2024-04-23T18:35:38.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.302+0000 7f5e82409700 1 -- 172.21.15.132:0/3827639613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e7c1019a0 msgr2=0x7f5e7c06b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:38.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.302+0000 7f5e82409700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e7c1019a0 0x7f5e7c06b2a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00e970 tx=0x7f5e6c00ec80 comp rx=0 tx=0).stop 2024-04-23T18:35:38.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.302+0000 7f5e82409700 1 -- 172.21.15.132:0/3827639613 shutdown_connections 2024-04-23T18:35:38.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.302+0000 7f5e82409700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7c103540 0x7f5e7c19ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:38.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.302+0000 7f5e82409700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f5e640a3cc0 0x7f5e640a6180 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:38.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.302+0000 7f5e82409700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e7c1019a0 0x7f5e7c06b2a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:38.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.302+0000 7f5e82409700 1 --2- 172.21.15.132:0/3827639613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e7c102ba0 0x7f5e7c19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:38.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.302+0000 7f5e82409700 1 -- 172.21.15.132:0/3827639613 >> 172.21.15.132:0/3827639613 conn(0x7f5e7c0fcf40 msgr2=0x7f5e7c105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:38.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.302+0000 7f5e82409700 1 -- 172.21.15.132:0/3827639613 shutdown_connections 2024-04-23T18:35:38.295 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:38.302+0000 7f5e82409700 1 -- 172.21.15.132:0/3827639613 wait complete. 2024-04-23T18:35:39.062 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-23T18:35:39.304 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:39.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:39 smithi132 ceph-mon[29178]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:35:39.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:39 smithi148 ceph-mon[32311]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:35:39.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:39 smithi157 ceph-mon[32580]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:35:40.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:40 smithi157 ceph-mon[32580]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:40.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:40 smithi132 ceph-mon[29178]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:40.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:40 smithi148 ceph-mon[32311]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:41.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.015+0000 7f018b457700 1 -- 172.21.15.132:0/1435346834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0184104640 msgr2=0x7f018410aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:41.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.015+0000 7f018b457700 1 --2- 172.21.15.132:0/1435346834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0184104640 0x7f018410aef0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f017c00b0d0 tx=0x7f017c00b3e0 comp rx=0 tx=0).stop 2024-04-23T18:35:41.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.015+0000 7f018b457700 1 -- 172.21.15.132:0/1435346834 shutdown_connections 2024-04-23T18:35:41.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.015+0000 7f018b457700 1 --2- 172.21.15.132:0/1435346834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0184104640 0x7f018410aef0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:41.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.015+0000 7f018b457700 1 --2- 172.21.15.132:0/1435346834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0184102aa0 0x7f0184102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:41.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.015+0000 7f018b457700 1 --2- 172.21.15.132:0/1435346834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0184103ca0 0x7f0184104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:41.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.015+0000 7f018b457700 1 -- 172.21.15.132:0/1435346834 >> 172.21.15.132:0/1435346834 conn(0x7f01840fe040 msgr2=0x7f0184100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f018b457700 1 -- 172.21.15.132:0/1435346834 shutdown_connections 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f018b457700 1 -- 172.21.15.132:0/1435346834 wait complete. 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f018b457700 1 Processor -- start 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f018b457700 1 -- start start 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f018b457700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0184102aa0 0x7f018419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f018b457700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0184103ca0 0x7f018419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f018b457700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0184104640 0x7f01841a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f018b457700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f018410e5b0 con 0x7f0184103ca0 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f018b457700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f018410e430 con 0x7f0184104640 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f018b457700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f018410e730 con 0x7f0184102aa0 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f01889f2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0184103ca0 0x7f018419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f01889f2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0184103ca0 0x7f018419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55740/0 (socket says 172.21.15.132:55740) 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.016+0000 7f01889f2700 1 -- 172.21.15.132:0/928913251 learned_addr learned my addr 172.21.15.132:0/928913251 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:41.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f01891f3700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0184102aa0 0x7f018419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:41.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f01889f2700 1 -- 172.21.15.132:0/928913251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0184102aa0 msgr2=0x7f018419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:41.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f01889f2700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0184102aa0 0x7f018419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:41.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f01899f4700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0184104640 0x7f01841a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:41.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f01889f2700 1 -- 172.21.15.132:0/928913251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0184104640 msgr2=0x7f01841a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:41.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f01889f2700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0184104640 0x7f01841a23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:41.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f01889f2700 1 -- 172.21.15.132:0/928913251 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01841a2a90 con 0x7f0184103ca0 2024-04-23T18:35:41.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f01889f2700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0184103ca0 0x7f018419b370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f018000c950 tx=0x7f018000cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:41.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f017a7fc700 1 -- 172.21.15.132:0/928913251 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180007ac0 con 0x7f0184103ca0 2024-04-23T18:35:41.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f017a7fc700 1 -- 172.21.15.132:0/928913251 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0180004ce0 con 0x7f0184103ca0 2024-04-23T18:35:41.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f018b457700 1 -- 172.21.15.132:0/928913251 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01841a2d80 con 0x7f0184103ca0 2024-04-23T18:35:41.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f017a7fc700 1 -- 172.21.15.132:0/928913251 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01800055a0 con 0x7f0184103ca0 2024-04-23T18:35:41.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.017+0000 7f018b457700 1 -- 172.21.15.132:0/928913251 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01841a32e0 con 0x7f0184103ca0 2024-04-23T18:35:41.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.018+0000 7f018b457700 1 -- 172.21.15.132:0/928913251 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0184069760 con 0x7f0184103ca0 2024-04-23T18:35:41.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.019+0000 7f017a7fc700 1 -- 172.21.15.132:0/928913251 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f0180007c60 con 0x7f0184103ca0 2024-04-23T18:35:41.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.020+0000 7f017a7fc700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f01700a3c40 0x7f01700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:41.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.020+0000 7f01891f3700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f01700a3c40 0x7f01700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:41.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.020+0000 7f017a7fc700 1 -- 172.21.15.132:0/928913251 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f01800d0460 con 0x7f0184103ca0 2024-04-23T18:35:41.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.020+0000 7f01891f3700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f01700a3c40 0x7f01700a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0174006fd0 tx=0x7f0174008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:41.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.023+0000 7f017a7fc700 1 -- 172.21.15.132:0/928913251 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0184069760 con 0x7f0184103ca0 2024-04-23T18:35:41.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.229+0000 7f018b457700 1 -- 172.21.15.132:0/928913251 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f0184076390 con 0x7f01700a3c40 2024-04-23T18:35:41.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.230+0000 7f017a7fc700 1 -- 172.21.15.132:0/928913251 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f0184076390 con 0x7f01700a3c40 2024-04-23T18:35:41.223 INFO:teuthology.orchestra.run.smithi132.stdout:HOST ADDR LABELS STATUS 2024-04-23T18:35:41.224 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132 172.21.15.132 2024-04-23T18:35:41.224 INFO:teuthology.orchestra.run.smithi132.stdout:smithi148 172.21.15.148 2024-04-23T18:35:41.224 INFO:teuthology.orchestra.run.smithi132.stdout:smithi157 172.21.15.157 2024-04-23T18:35:41.224 INFO:teuthology.orchestra.run.smithi132.stdout:3 hosts in cluster 2024-04-23T18:35:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.233+0000 7f018b457700 1 -- 172.21.15.132:0/928913251 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f01700a3c40 msgr2=0x7f01700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.233+0000 7f018b457700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f01700a3c40 0x7f01700a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0174006fd0 tx=0x7f0174008040 comp rx=0 tx=0).stop 2024-04-23T18:35:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.233+0000 7f018b457700 1 -- 172.21.15.132:0/928913251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0184103ca0 msgr2=0x7f018419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.233+0000 7f018b457700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0184103ca0 0x7f018419b370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f018000c950 tx=0x7f018000cd10 comp rx=0 tx=0).stop 2024-04-23T18:35:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.233+0000 7f018b457700 1 -- 172.21.15.132:0/928913251 shutdown_connections 2024-04-23T18:35:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.233+0000 7f018b457700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0184104640 0x7f01841a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.234+0000 7f018b457700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0184102aa0 0x7f018419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.234+0000 7f018b457700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f01700a3c40 0x7f01700a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.234+0000 7f018b457700 1 --2- 172.21.15.132:0/928913251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0184103ca0 0x7f018419b370 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.234+0000 7f018b457700 1 -- 172.21.15.132:0/928913251 >> 172.21.15.132:0/928913251 conn(0x7f01840fe040 msgr2=0x7f0184074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.234+0000 7f018b457700 1 -- 172.21.15.132:0/928913251 shutdown_connections 2024-04-23T18:35:41.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:41.234+0000 7f018b457700 1 -- 172.21.15.132:0/928913251 wait complete. 2024-04-23T18:35:41.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:41 smithi157 ceph-mon[32580]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T18:35:41.599 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:41 smithi132 ceph-mon[29178]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T18:35:41.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:41 smithi148 ceph-mon[32311]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T18:35:41.971 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-23T18:35:42.222 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:42.494 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:42 smithi132 ceph-mon[29178]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:42.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:42 smithi157 ceph-mon[32580]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:42.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:42 smithi148 ceph-mon[32311]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:43.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:43 smithi132 ceph-mon[29178]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:35:43.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:43 smithi157 ceph-mon[32580]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:35:43.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:43 smithi148 ceph-mon[32311]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:35:44.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.032+0000 7fba3cbed700 1 -- 172.21.15.132:0/2350810744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba38108780 msgr2=0x7fba3810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.032+0000 7fba3cbed700 1 --2- 172.21.15.132:0/2350810744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba38108780 0x7fba3810ac10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fba3000b0a0 tx=0x7fba3000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.032+0000 7fba3cbed700 1 -- 172.21.15.132:0/2350810744 shutdown_connections 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.032+0000 7fba3cbed700 1 --2- 172.21.15.132:0/2350810744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba38108780 0x7fba3810ac10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.032+0000 7fba3cbed700 1 --2- 172.21.15.132:0/2350810744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba38100700 0x7fba38100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.032+0000 7fba3cbed700 1 --2- 172.21.15.132:0/2350810744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba381010f0 0x7fba38108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.032+0000 7fba3cbed700 1 -- 172.21.15.132:0/2350810744 >> 172.21.15.132:0/2350810744 conn(0x7fba380fc290 msgr2=0x7fba380fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.032+0000 7fba3cbed700 1 -- 172.21.15.132:0/2350810744 shutdown_connections 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.032+0000 7fba3cbed700 1 -- 172.21.15.132:0/2350810744 wait complete. 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba3cbed700 1 Processor -- start 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba3cbed700 1 -- start start 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba3cbed700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba38100700 0x7fba38198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba3cbed700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba381010f0 0x7fba38199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:44.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba3cbed700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba38108780 0x7fba381a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba3cbed700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fba3810e5f0 con 0x7fba381010f0 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba3cbed700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fba3810e470 con 0x7fba38108780 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba3cbed700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba3810e770 con 0x7fba38100700 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba35d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba381010f0 0x7fba38199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba35d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba381010f0 0x7fba38199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55752/0 (socket says 172.21.15.132:55752) 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba35d9b700 1 -- 172.21.15.132:0/275855304 learned_addr learned my addr 172.21.15.132:0/275855304 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba3659c700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba38100700 0x7fba38198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.033+0000 7fba36d9d700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba38108780 0x7fba381a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.034+0000 7fba35d9b700 1 -- 172.21.15.132:0/275855304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba38100700 msgr2=0x7fba38198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.034+0000 7fba35d9b700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba38100700 0x7fba38198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.034+0000 7fba35d9b700 1 -- 172.21.15.132:0/275855304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba38108780 msgr2=0x7fba381a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.034+0000 7fba35d9b700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba38108780 0x7fba381a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:44.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.034+0000 7fba35d9b700 1 -- 172.21.15.132:0/275855304 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba381a08c0 con 0x7fba381010f0 2024-04-23T18:35:44.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.034+0000 7fba35d9b700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba381010f0 0x7fba38199250 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00b530 tx=0x7fba2c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:44.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.034+0000 7fba277fe700 1 -- 172.21.15.132:0/275855304 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c015020 con 0x7fba381010f0 2024-04-23T18:35:44.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.034+0000 7fba277fe700 1 -- 172.21.15.132:0/275855304 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba2c011ce0 con 0x7fba381010f0 2024-04-23T18:35:44.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.034+0000 7fba3cbed700 1 -- 172.21.15.132:0/275855304 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba381a0bb0 con 0x7fba381010f0 2024-04-23T18:35:44.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.034+0000 7fba277fe700 1 -- 172.21.15.132:0/275855304 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c010dd0 con 0x7fba381010f0 2024-04-23T18:35:44.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.034+0000 7fba3cbed700 1 -- 172.21.15.132:0/275855304 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba381a10f0 con 0x7fba381010f0 2024-04-23T18:35:44.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.035+0000 7fba3cbed700 1 -- 172.21.15.132:0/275855304 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba38069730 con 0x7fba381010f0 2024-04-23T18:35:44.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.037+0000 7fba277fe700 1 -- 172.21.15.132:0/275855304 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fba2c018020 con 0x7fba381010f0 2024-04-23T18:35:44.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.037+0000 7fba277fe700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fba200a3c40 0x7fba200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:44.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.038+0000 7fba3659c700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fba200a3c40 0x7fba200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:44.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.038+0000 7fba277fe700 1 -- 172.21.15.132:0/275855304 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fba2c0d0030 con 0x7fba381010f0 2024-04-23T18:35:44.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.038+0000 7fba3659c700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fba200a3c40 0x7fba200a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba3819a150 tx=0x7fba28009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:44.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.041+0000 7fba277fe700 1 -- 172.21.15.132:0/275855304 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba2c01f020 con 0x7fba381010f0 2024-04-23T18:35:44.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.250+0000 7fba3cbed700 1 -- 172.21.15.132:0/275855304 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fba38062e80 con 0x7fba200a3c40 2024-04-23T18:35:44.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.251+0000 7fba277fe700 1 -- 172.21.15.132:0/275855304 <== mgr.14152 v2:172.21.15.132:6800/636352941 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fba38062e80 con 0x7fba200a3c40 2024-04-23T18:35:44.243 INFO:teuthology.orchestra.run.smithi132.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-23T18:35:44.244 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232000B400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-23T18:35:44.244 INFO:teuthology.orchestra.run.smithi132.stdout:smithi148 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005G400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-23T18:35:44.244 INFO:teuthology.orchestra.run.smithi132.stdout:smithi157 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002V480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-04-23T18:35:44.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.254+0000 7fba3cbed700 1 -- 172.21.15.132:0/275855304 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fba200a3c40 msgr2=0x7fba200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:44.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.254+0000 7fba3cbed700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fba200a3c40 0x7fba200a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba3819a150 tx=0x7fba28009190 comp rx=0 tx=0).stop 2024-04-23T18:35:44.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.254+0000 7fba3cbed700 1 -- 172.21.15.132:0/275855304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba381010f0 msgr2=0x7fba38199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:44.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.254+0000 7fba3cbed700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba381010f0 0x7fba38199250 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00b530 tx=0x7fba2c00b840 comp rx=0 tx=0).stop 2024-04-23T18:35:44.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.254+0000 7fba3cbed700 1 -- 172.21.15.132:0/275855304 shutdown_connections 2024-04-23T18:35:44.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.254+0000 7fba3cbed700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba38108780 0x7fba381a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:44.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.254+0000 7fba3cbed700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba38100700 0x7fba38198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:44.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.254+0000 7fba3cbed700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fba200a3c40 0x7fba200a6100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:44.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.254+0000 7fba3cbed700 1 --2- 172.21.15.132:0/275855304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba381010f0 0x7fba38199250 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:44.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.254+0000 7fba3cbed700 1 -- 172.21.15.132:0/275855304 >> 172.21.15.132:0/275855304 conn(0x7fba380fc290 msgr2=0x7fba38104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:44.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.254+0000 7fba3cbed700 1 -- 172.21.15.132:0/275855304 shutdown_connections 2024-04-23T18:35:44.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:44.254+0000 7fba3cbed700 1 -- 172.21.15.132:0/275855304 wait complete. 2024-04-23T18:35:44.935 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T18:35:44.944 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-04-23T18:35:44.944 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-23T18:35:45.187 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:45.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:45 smithi157 ceph-mon[32580]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:45.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:45 smithi157 ceph-mon[32580]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:45.573 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:45 smithi132 ceph-mon[29178]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:45.573 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:45 smithi132 ceph-mon[29178]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:45.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:45 smithi148 ceph-mon[32311]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:45.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:45 smithi148 ceph-mon[32311]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:35:46.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.940+0000 7f6b18db3700 1 -- 172.21.15.132:0/3140117835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b14103d70 msgr2=0x7f6b141041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.940+0000 7f6b18db3700 1 --2- 172.21.15.132:0/3140117835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b14103d70 0x7f6b141041d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009990 tx=0x7f6b08009ca0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.940+0000 7f6b18db3700 1 -- 172.21.15.132:0/3140117835 shutdown_connections 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.940+0000 7f6b18db3700 1 --2- 172.21.15.132:0/3140117835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b14104710 0x7f6b1410afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.940+0000 7f6b18db3700 1 --2- 172.21.15.132:0/3140117835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b14100d90 0x7f6b141011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.940+0000 7f6b18db3700 1 --2- 172.21.15.132:0/3140117835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b14103d70 0x7f6b141041d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.940+0000 7f6b18db3700 1 -- 172.21.15.132:0/3140117835 >> 172.21.15.132:0/3140117835 conn(0x7f6b140fc6f0 msgr2=0x7f6b140feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.940+0000 7f6b18db3700 1 -- 172.21.15.132:0/3140117835 shutdown_connections 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.940+0000 7f6b18db3700 1 -- 172.21.15.132:0/3140117835 wait complete. 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.941+0000 7f6b18db3700 1 Processor -- start 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.941+0000 7f6b18db3700 1 -- start start 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.941+0000 7f6b18db3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b14100d90 0x7f6b14198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.941+0000 7f6b18db3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b14103d70 0x7f6b141991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.941+0000 7f6b18db3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b14104710 0x7f6b141a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.941+0000 7f6b18db3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410e620 con 0x7f6b14104710 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.941+0000 7f6b18db3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410e4a0 con 0x7f6b14103d70 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.941+0000 7f6b18db3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410e7a0 con 0x7f6b14100d90 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b12d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b14104710 0x7f6b141a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b12d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b14104710 0x7f6b141a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55764/0 (socket says 172.21.15.132:55764) 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b12d9d700 1 -- 172.21.15.132:0/3007985955 learned_addr learned my addr 172.21.15.132:0/3007985955 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b1259c700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b14100d90 0x7f6b14198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b11d9b700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b14103d70 0x7f6b141991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b12d9d700 1 -- 172.21.15.132:0/3007985955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b14100d90 msgr2=0x7f6b14198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b12d9d700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b14100d90 0x7f6b14198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b12d9d700 1 -- 172.21.15.132:0/3007985955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b14103d70 msgr2=0x7f6b141991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b12d9d700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b14103d70 0x7f6b141991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b12d9d700 1 -- 172.21.15.132:0/3007985955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b141a08f0 con 0x7f6b14104710 2024-04-23T18:35:46.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b12d9d700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b14104710 0x7f6b141a0220 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c007ee0 tx=0x7f6b0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b037fe700 1 -- 172.21.15.132:0/3007985955 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c004bb0 con 0x7f6b14104710 2024-04-23T18:35:46.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b037fe700 1 -- 172.21.15.132:0/3007985955 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b0c004d50 con 0x7f6b14104710 2024-04-23T18:35:46.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b18db3700 1 -- 172.21.15.132:0/3007985955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b141a0be0 con 0x7f6b14104710 2024-04-23T18:35:46.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b037fe700 1 -- 172.21.15.132:0/3007985955 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c005920 con 0x7f6b14104710 2024-04-23T18:35:46.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.942+0000 7f6b18db3700 1 -- 172.21.15.132:0/3007985955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b14101b60 con 0x7f6b14104710 2024-04-23T18:35:46.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.943+0000 7f6b18db3700 1 -- 172.21.15.132:0/3007985955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b14069730 con 0x7f6b14104710 2024-04-23T18:35:46.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.944+0000 7f6b037fe700 1 -- 172.21.15.132:0/3007985955 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f6b0c01a020 con 0x7f6b14104710 2024-04-23T18:35:46.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.945+0000 7f6b037fe700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6afc0a8050 0x7f6afc0aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:46.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.945+0000 7f6b1259c700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6afc0a8050 0x7f6afc0aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:46.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.945+0000 7f6b037fe700 1 -- 172.21.15.132:0/3007985955 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6b0c0d0170 con 0x7f6b14104710 2024-04-23T18:35:46.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.945+0000 7f6b1259c700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6afc0a8050 0x7f6afc0aa510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6b14199b50 tx=0x7f6b04009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:46.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:46.949+0000 7f6b037fe700 1 -- 172.21.15.132:0/3007985955 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b0c010070 con 0x7f6b14104710 2024-04-23T18:35:47.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.191+0000 7f6b18db3700 1 -- 172.21.15.132:0/3007985955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6b1402dfb0 con 0x7f6b14104710 2024-04-23T18:35:47.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.192+0000 7f6b037fe700 1 -- 172.21.15.132:0/3007985955 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3100 (secure 0 0 0) 0x7f6b0c09cb40 con 0x7f6b14104710 2024-04-23T18:35:47.184 INFO:teuthology.orchestra.run.smithi132.stdout:e25 2024-04-23T18:35:47.184 INFO:teuthology.orchestra.run.smithi132.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T18:35:47.184 INFO:teuthology.orchestra.run.smithi132.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:35:47.184 INFO:teuthology.orchestra.run.smithi132.stdout:legacy client fscid: 1 2024-04-23T18:35:47.184 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:47.184 INFO:teuthology.orchestra.run.smithi132.stdout:Filesystem 'cephfs' (1) 2024-04-23T18:35:47.184 INFO:teuthology.orchestra.run.smithi132.stdout:fs_name cephfs 2024-04-23T18:35:47.184 INFO:teuthology.orchestra.run.smithi132.stdout:epoch 25 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:created 2024-04-23T18:34:00.571339+0000 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:modified 2024-04-23T18:34:07.536766+0000 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:tableserver 0 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:root 0 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:session_timeout 300 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:session_autoclose 300 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:max_file_size 1099511627776 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:max_xattr_size 65536 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:required_client_features {} 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:last_failure 0 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:last_failure_osd_epoch 0 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:35:47.185 INFO:teuthology.orchestra.run.smithi132.stdout:max_mds 5 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:in 0,1,2,3,4 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:up {0=24515,1=24491,2=24460,3=24425,4=24394} 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:failed 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:damaged 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:stopped 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:data_pools [3] 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:metadata_pool 2 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:inline_data disabled 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:balancer 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:bal_rank_mask -1 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:standby_count_wanted 0 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:qdb_cluster leader: 24515 members: 24491,24515,24394,24425,24460 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b{0:24515} state up:active seq 3 addr [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:35:47.186 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.e{0:24458} state up:standby-replay seq 1 addr [v2:172.21.15.148:6834/397710093,v1:172.21.15.148:6835/397710093] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.h{1:24491} state up:active seq 7 addr [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c{1:24439} state up:standby-replay seq 1 addr [v2:172.21.15.157:6836/2670322328,v1:172.21.15.157:6837/2670322328] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.i{2:24460} state up:active seq 6 addr [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.f{2:24430} state up:standby-replay seq 1 addr [v2:172.21.15.157:6834/4121037781,v1:172.21.15.157:6835/4121037781] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.k{3:24425} state up:active seq 11 addr [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a{3:14640} state up:standby-replay seq 1 addr [v2:172.21.15.132:6840/826931131,v1:172.21.15.132:6841/826931131] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.l{4:24394} state up:active seq 12 addr [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.g{4:14628} state up:standby-replay seq 1 addr [v2:172.21.15.132:6838/579251026,v1:172.21.15.132:6839/579251026] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout:Standby daemons: 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.132:6834/152253190,v1:172.21.15.132:6835/152253190] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:35:47.187 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.132:6836/1545569950,v1:172.21.15.132:6837/1545569950] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:35:47.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.194+0000 7f6b18db3700 1 -- 172.21.15.132:0/3007985955 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6afc0a8050 msgr2=0x7f6afc0aa510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:47.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.194+0000 7f6b18db3700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6afc0a8050 0x7f6afc0aa510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6b14199b50 tx=0x7f6b04009190 comp rx=0 tx=0).stop 2024-04-23T18:35:47.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.194+0000 7f6b18db3700 1 -- 172.21.15.132:0/3007985955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b14104710 msgr2=0x7f6b141a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:47.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.194+0000 7f6b18db3700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b14104710 0x7f6b141a0220 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c007ee0 tx=0x7f6b0c00d3b0 comp rx=0 tx=0).stop 2024-04-23T18:35:47.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.194+0000 7f6b18db3700 1 -- 172.21.15.132:0/3007985955 shutdown_connections 2024-04-23T18:35:47.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.194+0000 7f6b18db3700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6afc0a8050 0x7f6afc0aa510 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:47.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.194+0000 7f6b18db3700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b14104710 0x7f6b141a0220 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:47.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.194+0000 7f6b18db3700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b14100d90 0x7f6b14198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:47.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.194+0000 7f6b18db3700 1 --2- 172.21.15.132:0/3007985955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b14103d70 0x7f6b141991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:47.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.194+0000 7f6b18db3700 1 -- 172.21.15.132:0/3007985955 >> 172.21.15.132:0/3007985955 conn(0x7f6b140fc6f0 msgr2=0x7f6b141070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:47.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.194+0000 7f6b18db3700 1 -- 172.21.15.132:0/3007985955 shutdown_connections 2024-04-23T18:35:47.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:47.194+0000 7f6b18db3700 1 -- 172.21.15.132:0/3007985955 wait complete. 2024-04-23T18:35:47.189 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 25 2024-04-23T18:35:47.512 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:47 smithi132 ceph-mon[29178]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:35:47.512 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:47 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3007985955' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T18:35:47.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:47 smithi157 ceph-mon[32580]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:35:47.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:47 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3007985955' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T18:35:47.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:47 smithi148 ceph-mon[32311]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T18:35:47.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:47 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3007985955' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T18:35:47.873 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-23T18:35:48.111 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:49.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:49 smithi157 ceph-mon[32580]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:49.588 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:49 smithi132 ceph-mon[29178]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:49.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:49 smithi148 ceph-mon[32311]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:49.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/903472695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfc8102a90 msgr2=0x7fdfc8102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:49.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 --2- 172.21.15.132:0/903472695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfc8102a90 0x7fdfc8102eb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80099c0 tx=0x7fdfb8009cd0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/903472695 shutdown_connections 2024-04-23T18:35:49.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 --2- 172.21.15.132:0/903472695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc8104630 0x7fdfc810aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 --2- 172.21.15.132:0/903472695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfc8102a90 0x7fdfc8102eb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 --2- 172.21.15.132:0/903472695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8103c90 0x7fdfc81040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:49.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/903472695 >> 172.21.15.132:0/903472695 conn(0x7fdfc80fe030 msgr2=0x7fdfc8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:49.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/903472695 shutdown_connections 2024-04-23T18:35:49.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/903472695 wait complete. 2024-04-23T18:35:49.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 Processor -- start 2024-04-23T18:35:49.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 -- start start 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfc8102a90 0x7fdfc819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc8103c90 0x7fdfc819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8104630 0x7fdfc81a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc810e500 con 0x7fdfc8102a90 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc810e380 con 0x7fdfc8103c90 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.005+0000 7fdfcd2ec700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc810e680 con 0x7fdfc8104630 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfc8102a90 0x7fdfc819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfc8102a90 0x7fdfc819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59900/0 (socket says 172.21.15.132:59900) 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc6ffd700 1 -- 172.21.15.132:0/3244102041 learned_addr learned my addr 172.21.15.132:0/3244102041 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfbffff700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc8103c90 0x7fdfc819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc77fe700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8104630 0x7fdfc81a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc6ffd700 1 -- 172.21.15.132:0/3244102041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8104630 msgr2=0x7fdfc81a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc6ffd700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8104630 0x7fdfc81a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:50.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc6ffd700 1 -- 172.21.15.132:0/3244102041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc8103c90 msgr2=0x7fdfc819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:50.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc6ffd700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc8103c90 0x7fdfc819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:50.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc6ffd700 1 -- 172.21.15.132:0/3244102041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc81a29b0 con 0x7fdfc8102a90 2024-04-23T18:35:50.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc77fe700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8104630 0x7fdfc81a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:50.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc6ffd700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfc8102a90 0x7fdfc819ad50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009f50 tx=0x7fdfb80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:50.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc4ff9700 1 -- 172.21.15.132:0/3244102041 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8027070 con 0x7fdfc8102a90 2024-04-23T18:35:50.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc4ff9700 1 -- 172.21.15.132:0/3244102041 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb80040a0 con 0x7fdfc8102a90 2024-04-23T18:35:50.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfc4ff9700 1 -- 172.21.15.132:0/3244102041 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8022da0 con 0x7fdfc8102a90 2024-04-23T18:35:50.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/3244102041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc81a2c40 con 0x7fdfc8102a90 2024-04-23T18:35:50.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.006+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/3244102041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc81a3170 con 0x7fdfc8102a90 2024-04-23T18:35:50.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.007+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/3244102041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc8069730 con 0x7fdfc8102a90 2024-04-23T18:35:50.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.008+0000 7fdfc4ff9700 1 -- 172.21.15.132:0/3244102041 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fdfb802a070 con 0x7fdfc8102a90 2024-04-23T18:35:50.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.009+0000 7fdfc4ff9700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdfb40a3cc0 0x7fdfb40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:50.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.009+0000 7fdfbffff700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdfb40a3cc0 0x7fdfb40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:50.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.009+0000 7fdfc4ff9700 1 -- 172.21.15.132:0/3244102041 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fdfb80d1590 con 0x7fdfc8102a90 2024-04-23T18:35:50.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.010+0000 7fdfbffff700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdfb40a3cc0 0x7fdfb40a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000ac50 tx=0x7fdfb000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:50.004 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.013+0000 7fdfc4ff9700 1 -- 172.21.15.132:0/3244102041 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfb8031070 con 0x7fdfc8102a90 2024-04-23T18:35:50.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.210+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/3244102041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fdfc81a3570 con 0x7fdfc8102a90 2024-04-23T18:35:50.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.211+0000 7fdfc4ff9700 1 -- 172.21.15.132:0/3244102041 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5348 (secure 0 0 0) 0x7fdfb8013070 con 0x7fdfc8102a90 2024-04-23T18:35:50.203 INFO:teuthology.orchestra.run.smithi132.stdout:epoch 88 2024-04-23T18:35:50.203 INFO:teuthology.orchestra.run.smithi132.stdout:fsid c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:35:50.203 INFO:teuthology.orchestra.run.smithi132.stdout:created 2024-04-23T18:26:00.784473+0000 2024-04-23T18:35:50.203 INFO:teuthology.orchestra.run.smithi132.stdout:modified 2024-04-23T18:34:55.248852+0000 2024-04-23T18:35:50.203 INFO:teuthology.orchestra.run.smithi132.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-23T18:35:50.203 INFO:teuthology.orchestra.run.smithi132.stdout:crush_version 26 2024-04-23T18:35:50.203 INFO:teuthology.orchestra.run.smithi132.stdout:full_ratio 0.95 2024-04-23T18:35:50.203 INFO:teuthology.orchestra.run.smithi132.stdout:backfillfull_ratio 0.9 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:nearfull_ratio 0.85 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:require_min_compat_client luminous 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:min_compat_client luminous 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:require_osd_release squid 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:stretch_mode_enabled false 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:max_osd 12 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6802/1754141150,v1:172.21.15.132:6803/1754141150] [v2:172.21.15.132:6804/1754141150,v1:172.21.15.132:6805/1754141150] exists,up 5387e5b6-e309-4bf9-abcd-bd917b437059 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6800/2532595181,v1:172.21.15.148:6801/2532595181] [v2:172.21.15.148:6802/2532595181,v1:172.21.15.148:6803/2532595181] exists,up 962fbcd8-143f-4f15-9e83-6afed0c5a72d 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6800/1299586857,v1:172.21.15.157:6801/1299586857] [v2:172.21.15.157:6802/1299586857,v1:172.21.15.157:6803/1299586857] exists,up 3c4885a5-c35c-4b55-bbd2-9370fc797096 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6810/2500771155,v1:172.21.15.132:6811/2500771155] [v2:172.21.15.132:6812/2500771155,v1:172.21.15.132:6813/2500771155] exists,up de1bb55b-fced-40bd-b2c3-81a227b2e891 2024-04-23T18:35:50.204 INFO:teuthology.orchestra.run.smithi132.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6808/1930596121,v1:172.21.15.148:6809/1930596121] [v2:172.21.15.148:6810/1930596121,v1:172.21.15.148:6811/1930596121] exists,up 0e84996c-3580-48d6-b38f-ad1cd3217e70 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6808/3700977653,v1:172.21.15.157:6809/3700977653] [v2:172.21.15.157:6810/3700977653,v1:172.21.15.157:6811/3700977653] exists,up f06f9283-8099-4922-8e1c-f28c4ea57a64 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:osd.6 up in weight 1 up_from 46 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6818/3702092226,v1:172.21.15.132:6819/3702092226] [v2:172.21.15.132:6820/3702092226,v1:172.21.15.132:6821/3702092226] exists,up 3b1233e4-b273-4c3e-89a4-ecec0758d4fe 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6816/199627413,v1:172.21.15.148:6817/199627413] [v2:172.21.15.148:6818/199627413,v1:172.21.15.148:6819/199627413] exists,up 93425a43-d24f-4456-81d6-239c59996a22 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6816/3005625159,v1:172.21.15.157:6817/3005625159] [v2:172.21.15.157:6818/3005625159,v1:172.21.15.157:6819/3005625159] exists,up f9285b86-b921-4e03-af87-4fd814b866cc 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6826/3598365445,v1:172.21.15.132:6827/3598365445] [v2:172.21.15.132:6828/3598365445,v1:172.21.15.132:6829/3598365445] exists,up 90c3d0c2-e221-41ac-8da7-b272837f6283 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6824/2494142299,v1:172.21.15.148:6825/2494142299] [v2:172.21.15.148:6826/2494142299,v1:172.21.15.148:6827/2494142299] exists,up e548975f-da5d-4394-a5c6-e658a6c89046 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6824/2341045818,v1:172.21.15.157:6825/2341045818] [v2:172.21.15.157:6826/2341045818,v1:172.21.15.157:6827/2341045818] exists,up 5d9c5bae-d624-411c-8636-8156934d51fe 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 2.4 [2,11] 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 2.10 [8,11] 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 2.1a [8,4] 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 2.1d [2,1] 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 3.3 [4,9] 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 3.1b [4,7] 2024-04-23T18:35:50.205 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 3.38 [3,9,5,8] 2024-04-23T18:35:50.206 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 3.39 [5,6] 2024-04-23T18:35:50.206 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 3.3a [4,8] 2024-04-23T18:35:50.206 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:0/4132015592 expires 2024-04-24T18:26:54.042392+0000 2024-04-23T18:35:50.206 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:6801/532999776 expires 2024-04-24T18:26:54.042392+0000 2024-04-23T18:35:50.206 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:0/1778322727 expires 2024-04-24T18:26:54.042392+0000 2024-04-23T18:35:50.206 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:0/1210637601 expires 2024-04-24T18:26:27.197140+0000 2024-04-23T18:35:50.206 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:6800/532999776 expires 2024-04-24T18:26:54.042392+0000 2024-04-23T18:35:50.206 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:6801/2811136616 expires 2024-04-24T18:26:27.197140+0000 2024-04-23T18:35:50.206 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:0/2746316554 expires 2024-04-24T18:26:27.197140+0000 2024-04-23T18:35:50.207 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:0/3819980873 expires 2024-04-24T18:26:27.197140+0000 2024-04-23T18:35:50.207 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:6800/2811136616 expires 2024-04-24T18:26:27.197140+0000 2024-04-23T18:35:50.207 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:0/1096830498 expires 2024-04-24T18:26:54.042392+0000 2024-04-23T18:35:50.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.214+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/3244102041 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdfb40a3cc0 msgr2=0x7fdfb40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:50.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.214+0000 7fdfcd2ec700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdfb40a3cc0 0x7fdfb40a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000ac50 tx=0x7fdfb000a380 comp rx=0 tx=0).stop 2024-04-23T18:35:50.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.214+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/3244102041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfc8102a90 msgr2=0x7fdfc819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:50.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.214+0000 7fdfcd2ec700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfc8102a90 0x7fdfc819ad50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009f50 tx=0x7fdfb80047b0 comp rx=0 tx=0).stop 2024-04-23T18:35:50.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.214+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/3244102041 shutdown_connections 2024-04-23T18:35:50.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.214+0000 7fdfcd2ec700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8104630 0x7fdfc81a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:50.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.214+0000 7fdfcd2ec700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fdfb40a3cc0 0x7fdfb40a6180 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:50.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.214+0000 7fdfcd2ec700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfc8102a90 0x7fdfc819ad50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:50.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.214+0000 7fdfcd2ec700 1 --2- 172.21.15.132:0/3244102041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc8103c90 0x7fdfc819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:50.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.214+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/3244102041 >> 172.21.15.132:0/3244102041 conn(0x7fdfc80fe030 msgr2=0x7fdfc8074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:50.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.214+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/3244102041 shutdown_connections 2024-04-23T18:35:50.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:50.214+0000 7fdfcd2ec700 1 -- 172.21.15.132:0/3244102041 wait complete. 2024-04-23T18:35:50.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:50 smithi157 ceph-mon[32580]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:35:50.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:50 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3244102041' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T18:35:50.581 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:50 smithi132 ceph-mon[29178]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:35:50.581 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:50 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3244102041' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T18:35:50.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:50 smithi148 ceph-mon[32311]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T18:35:50.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:50 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3244102041' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T18:35:50.943 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-23T18:35:50.955 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:35:51.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.064+0000 7f764d460640 1 -- 172.21.15.132:0/3851056766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7648101fa0 msgr2=0x7f76481023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.064+0000 7f764d460640 1 --2- 172.21.15.132:0/3851056766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7648101fa0 0x7f76481023c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7638009900 tx=0x7f763802a410 comp rx=0 tx=0).stop 2024-04-23T18:35:51.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.064+0000 7f764d460640 1 -- 172.21.15.132:0/3851056766 shutdown_connections 2024-04-23T18:35:51.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.064+0000 7f764d460640 1 --2- 172.21.15.132:0/3851056766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7648103b50 0x7f764810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.064+0000 7f764d460640 1 --2- 172.21.15.132:0/3851056766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7648103190 0x7f7648103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.064+0000 7f764d460640 1 --2- 172.21.15.132:0/3851056766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7648101fa0 0x7f76481023c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.064+0000 7f764d460640 1 -- 172.21.15.132:0/3851056766 >> 172.21.15.132:0/3851056766 conn(0x7f76480fd530 msgr2=0x7f76480ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.064+0000 7f764d460640 1 -- 172.21.15.132:0/3851056766 shutdown_connections 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.064+0000 7f764d460640 1 -- 172.21.15.132:0/3851056766 wait complete. 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f764d460640 1 Processor -- start 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f764d460640 1 -- start start 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f764d460640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7648101fa0 0x7f7648198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f764d460640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7648103190 0x7f7648198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f764d460640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7648103b50 0x7f764819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f764d460640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f764810db40 con 0x7f7648103190 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f764d460640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f764810d9c0 con 0x7f7648101fa0 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f764d460640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f764810dcc0 con 0x7f7648103b50 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f76467fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7648103190 0x7f7648198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f76467fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7648103190 0x7f7648198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59918/0 (socket says 172.21.15.132:59918) 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f76467fc640 1 -- 172.21.15.132:0/447092489 learned_addr learned my addr 172.21.15.132:0/447092489 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f76477fe640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7648103b50 0x7f764819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f7646ffd640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7648101fa0 0x7f7648198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f76467fc640 1 -- 172.21.15.132:0/447092489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7648103b50 msgr2=0x7f764819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f76467fc640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7648103b50 0x7f764819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f76467fc640 1 -- 172.21.15.132:0/447092489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7648101fa0 msgr2=0x7f7648198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f76467fc640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7648101fa0 0x7f7648198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f76467fc640 1 -- 172.21.15.132:0/447092489 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764819fe60 con 0x7f7648103190 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f76477fe640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7648103b50 0x7f764819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.065+0000 7f7646ffd640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7648101fa0 0x7f7648198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.066+0000 7f76467fc640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7648103190 0x7f7648198780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f763c00b4a0 tx=0x7f763c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.066+0000 7f762ffff640 1 -- 172.21.15.132:0/447092489 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c014070 con 0x7f7648103190 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.066+0000 7f762ffff640 1 -- 172.21.15.132:0/447092489 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763c010ac0 con 0x7f7648103190 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.066+0000 7f764d460640 1 -- 172.21.15.132:0/447092489 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76481a0150 con 0x7f7648103190 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.066+0000 7f762ffff640 1 -- 172.21.15.132:0/447092489 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c017c30 con 0x7f7648103190 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.066+0000 7f764d460640 1 -- 172.21.15.132:0/447092489 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481a0720 con 0x7f7648103190 2024-04-23T18:35:51.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.067+0000 7f764d460640 1 -- 172.21.15.132:0/447092489 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7648109310 con 0x7f7648103190 2024-04-23T18:35:51.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.068+0000 7f762ffff640 1 -- 172.21.15.132:0/447092489 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f763c017070 con 0x7f7648103190 2024-04-23T18:35:51.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.068+0000 7f762ffff640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f76300a3bb0 0x7f76300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.069+0000 7f7646ffd640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f76300a3bb0 0x7f76300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.069+0000 7f762ffff640 1 -- 172.21.15.132:0/447092489 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f763c0cfba0 con 0x7f7648103190 2024-04-23T18:35:51.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.069+0000 7f7646ffd640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f76300a3bb0 0x7f76300a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7638006010 tx=0x7f7638000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:51.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.073+0000 7f762ffff640 1 -- 172.21.15.132:0/447092489 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f763c09c570 con 0x7f7648103190 2024-04-23T18:35:51.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.251+0000 7f764d460640 1 -- 172.21.15.132:0/447092489 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7648062b50 con 0x7f7648103190 2024-04-23T18:35:51.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.252+0000 7f762ffff640 1 -- 172.21.15.132:0/447092489 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22800 (secure 0 0 0) 0x7f763c09be70 con 0x7f7648103190 2024-04-23T18:35:51.245 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:51.245 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":88,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:34:55.248852+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:35:51.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.254+0000 7f764d460640 1 -- 172.21.15.132:0/447092489 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f76300a3bb0 msgr2=0x7f76300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.254+0000 7f764d460640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f76300a3bb0 0x7f76300a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7638006010 tx=0x7f7638000bc0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.254+0000 7f764d460640 1 -- 172.21.15.132:0/447092489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7648103190 msgr2=0x7f7648198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.254+0000 7f764d460640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7648103190 0x7f7648198780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f763c00b4a0 tx=0x7f763c00b960 comp rx=0 tx=0).stop 2024-04-23T18:35:51.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.254+0000 7f764d460640 1 -- 172.21.15.132:0/447092489 shutdown_connections 2024-04-23T18:35:51.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.254+0000 7f764d460640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f76300a3bb0 0x7f76300a6070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.254+0000 7f764d460640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7648103b50 0x7f764819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.254+0000 7f764d460640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7648103190 0x7f7648198780 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.254+0000 7f764d460640 1 --2- 172.21.15.132:0/447092489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7648101fa0 0x7f7648198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.254+0000 7f764d460640 1 -- 172.21.15.132:0/447092489 >> 172.21.15.132:0/447092489 conn(0x7f76480fd530 msgr2=0x7f76481064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:51.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.255+0000 7f764d460640 1 -- 172.21.15.132:0/447092489 shutdown_connections 2024-04-23T18:35:51.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.255+0000 7f764d460640 1 -- 172.21.15.132:0/447092489 wait complete. 2024-04-23T18:35:51.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:29:20.132451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:33:58.407521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:33:59.518206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:35:51.259 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:35:51.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.368+0000 7fc80c537640 1 -- 172.21.15.132:0/47885598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804103190 msgr2=0x7fc804103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.368+0000 7fc80c537640 1 --2- 172.21.15.132:0/47885598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804103190 0x7fc804103610 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009900 tx=0x7fc7f402a410 comp rx=0 tx=0).stop 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.368+0000 7fc80c537640 1 -- 172.21.15.132:0/47885598 shutdown_connections 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.368+0000 7fc80c537640 1 --2- 172.21.15.132:0/47885598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804103b50 0x7fc80410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.368+0000 7fc80c537640 1 --2- 172.21.15.132:0/47885598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804103190 0x7fc804103610 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.368+0000 7fc80c537640 1 --2- 172.21.15.132:0/47885598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804101fa0 0x7fc8041023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.368+0000 7fc80c537640 1 -- 172.21.15.132:0/47885598 >> 172.21.15.132:0/47885598 conn(0x7fc8040fd530 msgr2=0x7fc8040ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.368+0000 7fc80c537640 1 -- 172.21.15.132:0/47885598 shutdown_connections 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.368+0000 7fc80c537640 1 -- 172.21.15.132:0/47885598 wait complete. 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80c537640 1 Processor -- start 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80c537640 1 -- start start 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80c537640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804101fa0 0x7fc804198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80c537640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804103190 0x7fc804198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80c537640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804103b50 0x7fc80419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80c537640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc80410dae0 con 0x7fc804103b50 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80c537640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc80410d960 con 0x7fc804103190 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80c537640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc80410dc60 con 0x7fc804101fa0 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80aad4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804103b50 0x7fc80419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80aad4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804103b50 0x7fc80419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59950/0 (socket says 172.21.15.132:59950) 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80aad4640 1 -- 172.21.15.132:0/2079904825 learned_addr learned my addr 172.21.15.132:0/2079904825 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80a2d3640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804101fa0 0x7fc804198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc809ad2640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804103190 0x7fc804198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80aad4640 1 -- 172.21.15.132:0/2079904825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804101fa0 msgr2=0x7fc804198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80aad4640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804101fa0 0x7fc804198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80aad4640 1 -- 172.21.15.132:0/2079904825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804103190 msgr2=0x7fc804198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80aad4640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804103190 0x7fc804198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80aad4640 1 -- 172.21.15.132:0/2079904825 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80419fe40 con 0x7fc804103b50 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc809ad2640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804103190 0x7fc804198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:51.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.369+0000 7fc80aad4640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804103b50 0x7fc80419f7d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009fd0 tx=0x7fc7fc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:51.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.370+0000 7fc7fb7fe640 1 -- 172.21.15.132:0/2079904825 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc017070 con 0x7fc804103b50 2024-04-23T18:35:51.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.370+0000 7fc7fb7fe640 1 -- 172.21.15.132:0/2079904825 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc003e20 con 0x7fc804103b50 2024-04-23T18:35:51.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.370+0000 7fc80c537640 1 -- 172.21.15.132:0/2079904825 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8041a0130 con 0x7fc804103b50 2024-04-23T18:35:51.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.370+0000 7fc7fb7fe640 1 -- 172.21.15.132:0/2079904825 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc010460 con 0x7fc804103b50 2024-04-23T18:35:51.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.370+0000 7fc80c537640 1 -- 172.21.15.132:0/2079904825 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8041a0630 con 0x7fc804103b50 2024-04-23T18:35:51.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.370+0000 7fc80c537640 1 -- 172.21.15.132:0/2079904825 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8041093d0 con 0x7fc804103b50 2024-04-23T18:35:51.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.372+0000 7fc7fb7fe640 1 -- 172.21.15.132:0/2079904825 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fc7fc003970 con 0x7fc804103b50 2024-04-23T18:35:51.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.372+0000 7fc7fb7fe640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc7f00a3bb0 0x7fc7f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.372+0000 7fc80a2d3640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc7f00a3bb0 0x7fc7f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.372+0000 7fc7fb7fe640 1 -- 172.21.15.132:0/2079904825 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fc7fc0d0140 con 0x7fc804103b50 2024-04-23T18:35:51.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.373+0000 7fc80a2d3640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc7f00a3bb0 0x7fc7f00a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc800005fd0 tx=0x7fc80000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:51.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.376+0000 7fc7fb7fe640 1 -- 172.21.15.132:0/2079904825 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7fc09cb10 con 0x7fc804103b50 2024-04-23T18:35:51.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.553+0000 7fc80c537640 1 -- 172.21.15.132:0/2079904825 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc80402d5c0 con 0x7fc804103b50 2024-04-23T18:35:51.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.553+0000 7fc7fb7fe640 1 -- 172.21.15.132:0/2079904825 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fc7fc09c410 con 0x7fc804103b50 2024-04-23T18:35:51.546 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-04-23T18:35:51.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.555+0000 7fc80c537640 1 -- 172.21.15.132:0/2079904825 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc7f00a3bb0 msgr2=0x7fc7f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.555+0000 7fc80c537640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc7f00a3bb0 0x7fc7f00a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc800005fd0 tx=0x7fc80000b390 comp rx=0 tx=0).stop 2024-04-23T18:35:51.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.555+0000 7fc80c537640 1 -- 172.21.15.132:0/2079904825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804103b50 msgr2=0x7fc80419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.555+0000 7fc80c537640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804103b50 0x7fc80419f7d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009fd0 tx=0x7fc7fc00b260 comp rx=0 tx=0).stop 2024-04-23T18:35:51.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.555+0000 7fc80c537640 1 -- 172.21.15.132:0/2079904825 shutdown_connections 2024-04-23T18:35:51.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.555+0000 7fc80c537640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc7f00a3bb0 0x7fc7f00a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.555+0000 7fc80c537640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc804103b50 0x7fc80419f7d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.555+0000 7fc80c537640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc804103190 0x7fc804198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.555+0000 7fc80c537640 1 --2- 172.21.15.132:0/2079904825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc804101fa0 0x7fc804198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.555+0000 7fc80c537640 1 -- 172.21.15.132:0/2079904825 >> 172.21.15.132:0/2079904825 conn(0x7fc8040fd530 msgr2=0x7fc8041065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:51.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.555+0000 7fc80c537640 1 -- 172.21.15.132:0/2079904825 shutdown_connections 2024-04-23T18:35:51.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.555+0000 7fc80c537640 1 -- 172.21.15.132:0/2079904825 wait complete. 2024-04-23T18:35:51.559 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:35:51.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.668+0000 7f985f6ae640 1 -- 172.21.15.132:0/2487748317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9858101f80 msgr2=0x7f98581023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.668+0000 7f985f6ae640 1 --2- 172.21.15.132:0/2487748317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9858101f80 0x7f98581023a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9854009900 tx=0x7f985402a410 comp rx=0 tx=0).stop 2024-04-23T18:35:51.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.668+0000 7f985f6ae640 1 -- 172.21.15.132:0/2487748317 shutdown_connections 2024-04-23T18:35:51.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.668+0000 7f985f6ae640 1 --2- 172.21.15.132:0/2487748317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858103170 0x7f98581035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.668+0000 7f985f6ae640 1 --2- 172.21.15.132:0/2487748317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9858103b30 0x7f985810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.668+0000 7f985f6ae640 1 --2- 172.21.15.132:0/2487748317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9858101f80 0x7f98581023a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.668+0000 7f985f6ae640 1 -- 172.21.15.132:0/2487748317 >> 172.21.15.132:0/2487748317 conn(0x7f98580fd510 msgr2=0x7f98580ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.668+0000 7f985f6ae640 1 -- 172.21.15.132:0/2487748317 shutdown_connections 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.668+0000 7f985f6ae640 1 -- 172.21.15.132:0/2487748317 wait complete. 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.668+0000 7f985f6ae640 1 Processor -- start 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.668+0000 7f985f6ae640 1 -- start start 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985f6ae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9858101f80 0x7f98581981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985f6ae640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9858103170 0x7f9858198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985f6ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858103b30 0x7f985819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985f6ae640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f985810dac0 con 0x7f9858103170 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985f6ae640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f985810d940 con 0x7f9858101f80 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985f6ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f985810dc40 con 0x7f9858103b30 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985cc49640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9858103170 0x7f9858198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985cc49640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9858103170 0x7f9858198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59962/0 (socket says 172.21.15.132:59962) 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985cc49640 1 -- 172.21.15.132:0/634104928 learned_addr learned my addr 172.21.15.132:0/634104928 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985d44a640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9858101f80 0x7f98581981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985dc4b640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858103b30 0x7f985819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985cc49640 1 -- 172.21.15.132:0/634104928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858103b30 msgr2=0x7f985819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985cc49640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858103b30 0x7f985819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985cc49640 1 -- 172.21.15.132:0/634104928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9858101f80 msgr2=0x7f98581981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985cc49640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9858101f80 0x7f98581981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985cc49640 1 -- 172.21.15.132:0/634104928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985819fdf0 con 0x7f9858103170 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985d44a640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9858101f80 0x7f98581981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985cc49640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9858103170 0x7f9858198710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f984800d770 tx=0x7f984800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f984e7fc640 1 -- 172.21.15.132:0/634104928 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848017070 con 0x7f9858103170 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985f6ae640 1 -- 172.21.15.132:0/634104928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98581a00e0 con 0x7f9858103170 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f984e7fc640 1 -- 172.21.15.132:0/634104928 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98480054d0 con 0x7f9858103170 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f984e7fc640 1 -- 172.21.15.132:0/634104928 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848004c30 con 0x7f9858103170 2024-04-23T18:35:51.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.669+0000 7f985f6ae640 1 -- 172.21.15.132:0/634104928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98581a05e0 con 0x7f9858103170 2024-04-23T18:35:51.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.670+0000 7f985f6ae640 1 -- 172.21.15.132:0/634104928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98581092c0 con 0x7f9858103170 2024-04-23T18:35:51.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.671+0000 7f984e7fc640 1 -- 172.21.15.132:0/634104928 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f9848004dd0 con 0x7f9858103170 2024-04-23T18:35:51.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.672+0000 7f984e7fc640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98440a3bb0 0x7f98440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.672+0000 7f985d44a640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98440a3bb0 0x7f98440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.672+0000 7f984e7fc640 1 -- 172.21.15.132:0/634104928 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f98480d03c0 con 0x7f9858103170 2024-04-23T18:35:51.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.672+0000 7f985d44a640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98440a3bb0 0x7f98440a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9854000c00 tx=0x7f98540314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:51.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.676+0000 7f984e7fc640 1 -- 172.21.15.132:0/634104928 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f984809cd90 con 0x7f9858103170 2024-04-23T18:35:51.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:51 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/447092489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:51.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.855+0000 7f985f6ae640 1 -- 172.21.15.132:0/634104928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f985802d810 con 0x7f9858103170 2024-04-23T18:35:51.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.855+0000 7f984e7fc640 1 -- 172.21.15.132:0/634104928 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f984809c690 con 0x7f9858103170 2024-04-23T18:35:51.847 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-04-23T18:35:51.849 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.857+0000 7f985f6ae640 1 -- 172.21.15.132:0/634104928 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98440a3bb0 msgr2=0x7f98440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.849 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.857+0000 7f985f6ae640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98440a3bb0 0x7f98440a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9854000c00 tx=0x7f98540314a0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.857+0000 7f985f6ae640 1 -- 172.21.15.132:0/634104928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9858103170 msgr2=0x7f9858198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.857+0000 7f985f6ae640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9858103170 0x7f9858198710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f984800d770 tx=0x7f984800dc30 comp rx=0 tx=0).stop 2024-04-23T18:35:51.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.857+0000 7f985f6ae640 1 -- 172.21.15.132:0/634104928 shutdown_connections 2024-04-23T18:35:51.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.857+0000 7f985f6ae640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98440a3bb0 0x7f98440a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.857+0000 7f985f6ae640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9858103b30 0x7f985819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.857+0000 7f985f6ae640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9858103170 0x7f9858198710 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.857+0000 7f985f6ae640 1 --2- 172.21.15.132:0/634104928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9858101f80 0x7f98581981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.857+0000 7f985f6ae640 1 -- 172.21.15.132:0/634104928 >> 172.21.15.132:0/634104928 conn(0x7f98580fd510 msgr2=0x7f9858106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:51.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.857+0000 7f985f6ae640 1 -- 172.21.15.132:0/634104928 shutdown_connections 2024-04-23T18:35:51.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.857+0000 7f985f6ae640 1 -- 172.21.15.132:0/634104928 wait complete. 2024-04-23T18:35:51.859 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:51 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/447092489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:51.861 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:35:51.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.969+0000 7fc958cd9640 1 -- 172.21.15.132:0/829215977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc954102a90 msgr2=0x7fc954104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.969+0000 7fc958cd9640 1 --2- 172.21.15.132:0/829215977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc954102a90 0x7fc954104e80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc948009900 tx=0x7fc94802a410 comp rx=0 tx=0).stop 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 -- 172.21.15.132:0/829215977 shutdown_connections 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 --2- 172.21.15.132:0/829215977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9541053c0 0x7fc9541077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 --2- 172.21.15.132:0/829215977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc954107cf0 0x7fc95410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 --2- 172.21.15.132:0/829215977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc954102a90 0x7fc954104e80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 -- 172.21.15.132:0/829215977 >> 172.21.15.132:0/829215977 conn(0x7fc9540fc400 msgr2=0x7fc9540fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 -- 172.21.15.132:0/829215977 shutdown_connections 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 -- 172.21.15.132:0/829215977 wait complete. 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 Processor -- start 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 -- start start 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc954102a90 0x7fc9541982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9541053c0 0x7fc9541987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc954107cf0 0x7fc95419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc95410db30 con 0x7fc954107cf0 2024-04-23T18:35:51.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc95410d9b0 con 0x7fc954102a90 2024-04-23T18:35:51.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.970+0000 7fc958cd9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc95410dcb0 con 0x7fc9541053c0 2024-04-23T18:35:51.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc952d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc954107cf0 0x7fc95419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc952d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc954107cf0 0x7fc95419f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59984/0 (socket says 172.21.15.132:59984) 2024-04-23T18:35:51.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc952d9d640 1 -- 172.21.15.132:0/1915297192 learned_addr learned my addr 172.21.15.132:0/1915297192 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:51.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc951d9b640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9541053c0 0x7fc9541987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc952d9d640 1 -- 172.21.15.132:0/1915297192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9541053c0 msgr2=0x7fc9541987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc95259c640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc954102a90 0x7fc9541982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc952d9d640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9541053c0 0x7fc9541987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc952d9d640 1 -- 172.21.15.132:0/1915297192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc954102a90 msgr2=0x7fc9541982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:51.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc952d9d640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc954102a90 0x7fc9541982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:51.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc952d9d640 1 -- 172.21.15.132:0/1915297192 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95419fed0 con 0x7fc954107cf0 2024-04-23T18:35:51.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc95259c640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc954102a90 0x7fc9541982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:51.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc952d9d640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc954107cf0 0x7fc95419f860 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00cad0 tx=0x7fc94c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:51.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc9437fe640 1 -- 172.21.15.132:0/1915297192 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00ae20 con 0x7fc954107cf0 2024-04-23T18:35:51.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc9437fe640 1 -- 172.21.15.132:0/1915297192 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc94c005020 con 0x7fc954107cf0 2024-04-23T18:35:51.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc958cd9640 1 -- 172.21.15.132:0/1915297192 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9541a01c0 con 0x7fc954107cf0 2024-04-23T18:35:51.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc9437fe640 1 -- 172.21.15.132:0/1915297192 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00f6b0 con 0x7fc954107cf0 2024-04-23T18:35:51.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.971+0000 7fc958cd9640 1 -- 172.21.15.132:0/1915297192 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9541a0740 con 0x7fc954107cf0 2024-04-23T18:35:51.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.972+0000 7fc958cd9640 1 -- 172.21.15.132:0/1915297192 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9540fdef0 con 0x7fc954107cf0 2024-04-23T18:35:51.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.973+0000 7fc9437fe640 1 -- 172.21.15.132:0/1915297192 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fc94c0051d0 con 0x7fc954107cf0 2024-04-23T18:35:51.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.974+0000 7fc9437fe640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc93c0a3bb0 0x7fc93c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:51.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.974+0000 7fc95259c640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc93c0a3bb0 0x7fc93c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:51.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.974+0000 7fc9437fe640 1 -- 172.21.15.132:0/1915297192 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fc94c0d0580 con 0x7fc954107cf0 2024-04-23T18:35:51.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.974+0000 7fc95259c640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc93c0a3bb0 0x7fc93c0a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc948006010 tx=0x7fc948000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:51.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:51.978+0000 7fc9437fe640 1 -- 172.21.15.132:0/1915297192 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc94c014030 con 0x7fc954107cf0 2024-04-23T18:35:52.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:51 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/447092489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:52.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.157+0000 7fc958cd9640 1 -- 172.21.15.132:0/1915297192 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc954062b50 con 0x7fc954107cf0 2024-04-23T18:35:52.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.157+0000 7fc9437fe640 1 -- 172.21.15.132:0/1915297192 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fc94c09cf50 con 0x7fc954107cf0 2024-04-23T18:35:52.149 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-04-23T18:35:52.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.159+0000 7fc958cd9640 1 -- 172.21.15.132:0/1915297192 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc93c0a3bb0 msgr2=0x7fc93c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.159+0000 7fc958cd9640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc93c0a3bb0 0x7fc93c0a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc948006010 tx=0x7fc948000bc0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.159+0000 7fc958cd9640 1 -- 172.21.15.132:0/1915297192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc954107cf0 msgr2=0x7fc95419f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.159+0000 7fc958cd9640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc954107cf0 0x7fc95419f860 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00cad0 tx=0x7fc94c00cf90 comp rx=0 tx=0).stop 2024-04-23T18:35:52.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.159+0000 7fc958cd9640 1 -- 172.21.15.132:0/1915297192 shutdown_connections 2024-04-23T18:35:52.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.159+0000 7fc958cd9640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc93c0a3bb0 0x7fc93c0a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.159+0000 7fc958cd9640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9541053c0 0x7fc9541987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.159+0000 7fc958cd9640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc954107cf0 0x7fc95419f860 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.159+0000 7fc958cd9640 1 --2- 172.21.15.132:0/1915297192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc954102a90 0x7fc9541982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.159+0000 7fc958cd9640 1 -- 172.21.15.132:0/1915297192 >> 172.21.15.132:0/1915297192 conn(0x7fc9540fc400 msgr2=0x7fc954104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:52.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.159+0000 7fc958cd9640 1 -- 172.21.15.132:0/1915297192 shutdown_connections 2024-04-23T18:35:52.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.159+0000 7fc958cd9640 1 -- 172.21.15.132:0/1915297192 wait complete. 2024-04-23T18:35:52.163 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:35:52.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.270+0000 7f6440c91640 1 -- 172.21.15.132:0/424983825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f643c100e60 msgr2=0x7f643c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.270+0000 7f6440c91640 1 --2- 172.21.15.132:0/424983825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f643c100e60 0x7f643c101280 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f642c009900 tx=0x7f642c02a410 comp rx=0 tx=0).stop 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.270+0000 7f6440c91640 1 -- 172.21.15.132:0/424983825 shutdown_connections 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.270+0000 7f6440c91640 1 --2- 172.21.15.132:0/424983825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c102040 0x7f643c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.270+0000 7f6440c91640 1 --2- 172.21.15.132:0/424983825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643c102a00 0x7f643c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.270+0000 7f6440c91640 1 --2- 172.21.15.132:0/424983825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f643c100e60 0x7f643c101280 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.270+0000 7f6440c91640 1 -- 172.21.15.132:0/424983825 >> 172.21.15.132:0/424983825 conn(0x7f643c0fc440 msgr2=0x7f643c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.271+0000 7f6440c91640 1 -- 172.21.15.132:0/424983825 shutdown_connections 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.271+0000 7f6440c91640 1 -- 172.21.15.132:0/424983825 wait complete. 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.271+0000 7f6440c91640 1 Processor -- start 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.271+0000 7f6440c91640 1 -- start start 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.271+0000 7f6440c91640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643c100e60 0x7f643c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.271+0000 7f6440c91640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c102040 0x7f643c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.271+0000 7f6440c91640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f643c102a00 0x7f643c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:52.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.271+0000 7f6440c91640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10c9f0 con 0x7f643c102a00 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.271+0000 7f6440c91640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10c870 con 0x7f643c100e60 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.271+0000 7f6440c91640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10cb70 con 0x7f643c102040 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f643ad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f643c102a00 0x7f643c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f643ad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f643c102a00 0x7f643c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60012/0 (socket says 172.21.15.132:60012) 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f643ad9d640 1 -- 172.21.15.132:0/1085750503 learned_addr learned my addr 172.21.15.132:0/1085750503 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f6439d9b640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c102040 0x7f643c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f643ad9d640 1 -- 172.21.15.132:0/1085750503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c102040 msgr2=0x7f643c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f643a59c640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643c100e60 0x7f643c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f643ad9d640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c102040 0x7f643c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f643ad9d640 1 -- 172.21.15.132:0/1085750503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643c100e60 msgr2=0x7f643c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f643ad9d640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643c100e60 0x7f643c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f643ad9d640 1 -- 172.21.15.132:0/1085750503 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c19fe90 con 0x7f643c102a00 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f6439d9b640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c102040 0x7f643c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:52.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f643a59c640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643c100e60 0x7f643c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:52.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f643ad9d640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f643c102a00 0x7f643c19f820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f643400eb10 tx=0x7f643400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:52.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f642b7fe640 1 -- 172.21.15.132:0/1085750503 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434004b20 con 0x7f643c102a00 2024-04-23T18:35:52.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f6440c91640 1 -- 172.21.15.132:0/1085750503 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c1a0180 con 0x7f643c102a00 2024-04-23T18:35:52.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f642b7fe640 1 -- 172.21.15.132:0/1085750503 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6434004cc0 con 0x7f643c102a00 2024-04-23T18:35:52.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f642b7fe640 1 -- 172.21.15.132:0/1085750503 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434010670 con 0x7f643c102a00 2024-04-23T18:35:52.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.272+0000 7f6440c91640 1 -- 172.21.15.132:0/1085750503 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c1a0680 con 0x7f643c102a00 2024-04-23T18:35:52.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.273+0000 7f6440c91640 1 -- 172.21.15.132:0/1085750503 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c1a0940 con 0x7f643c102a00 2024-04-23T18:35:52.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.274+0000 7f642b7fe640 1 -- 172.21.15.132:0/1085750503 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f6434005020 con 0x7f643c102a00 2024-04-23T18:35:52.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.275+0000 7f642b7fe640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f64240a3bb0 0x7f64240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:52.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.275+0000 7f643a59c640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f64240a3bb0 0x7f64240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:52.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.275+0000 7f642b7fe640 1 -- 172.21.15.132:0/1085750503 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f64340d0af0 con 0x7f643c102a00 2024-04-23T18:35:52.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.275+0000 7f643a59c640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f64240a3bb0 0x7f64240a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f642c006010 tx=0x7f642c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:52.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.279+0000 7f642b7fe640 1 -- 172.21.15.132:0/1085750503 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f643409d4c0 con 0x7f643c102a00 2024-04-23T18:35:52.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.492+0000 7f6440c91640 1 -- 172.21.15.132:0/1085750503 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f643c1a0b10 con 0x7f643c102a00 2024-04-23T18:35:52.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.493+0000 7f642b7fe640 1 -- 172.21.15.132:0/1085750503 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11946 (secure 0 0 0) 0x7f643409cdc0 con 0x7f643c102a00 2024-04-23T18:35:52.485 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:52.486 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:34:07.536766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24394,24425,24460]},"id":1}]} 2024-04-23T18:35:52.486 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 25 2024-04-23T18:35:52.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.495+0000 7f6440c91640 1 -- 172.21.15.132:0/1085750503 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f64240a3bb0 msgr2=0x7f64240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.495+0000 7f6440c91640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f64240a3bb0 0x7f64240a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f642c006010 tx=0x7f642c000bc0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.495+0000 7f6440c91640 1 -- 172.21.15.132:0/1085750503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f643c102a00 msgr2=0x7f643c19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.495+0000 7f6440c91640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f643c102a00 0x7f643c19f820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f643400eb10 tx=0x7f643400c2d0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.495+0000 7f6440c91640 1 -- 172.21.15.132:0/1085750503 shutdown_connections 2024-04-23T18:35:52.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.495+0000 7f6440c91640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f64240a3bb0 0x7f64240a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.495+0000 7f6440c91640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c102040 0x7f643c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.495+0000 7f6440c91640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f643c102a00 0x7f643c19f820 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.495+0000 7f6440c91640 1 --2- 172.21.15.132:0/1085750503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643c100e60 0x7f643c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.495+0000 7f6440c91640 1 -- 172.21.15.132:0/1085750503 >> 172.21.15.132:0/1085750503 conn(0x7f643c0fc440 msgr2=0x7f643c06a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:52.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.496+0000 7f6440c91640 1 -- 172.21.15.132:0/1085750503 shutdown_connections 2024-04-23T18:35:52.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.496+0000 7f6440c91640 1 -- 172.21.15.132:0/1085750503 wait complete. 2024-04-23T18:35:52.499 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:35:52.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.606+0000 7f9870120640 1 -- 172.21.15.132:0/3215708133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9868101fa0 msgr2=0x7f98681023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.606+0000 7f9870120640 1 --2- 172.21.15.132:0/3215708133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9868101fa0 0x7f98681023c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9864009900 tx=0x7f986402a410 comp rx=0 tx=0).stop 2024-04-23T18:35:52.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.606+0000 7f9870120640 1 -- 172.21.15.132:0/3215708133 shutdown_connections 2024-04-23T18:35:52.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.606+0000 7f9870120640 1 --2- 172.21.15.132:0/3215708133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868103b50 0x7f986810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.606+0000 7f9870120640 1 --2- 172.21.15.132:0/3215708133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9868103190 0x7f9868103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.606+0000 7f9870120640 1 --2- 172.21.15.132:0/3215708133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9868101fa0 0x7f98681023c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.606+0000 7f9870120640 1 -- 172.21.15.132:0/3215708133 >> 172.21.15.132:0/3215708133 conn(0x7f98680fd530 msgr2=0x7f98680ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:52.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.606+0000 7f9870120640 1 -- 172.21.15.132:0/3215708133 shutdown_connections 2024-04-23T18:35:52.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.606+0000 7f9870120640 1 -- 172.21.15.132:0/3215708133 wait complete. 2024-04-23T18:35:52.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f9870120640 1 Processor -- start 2024-04-23T18:35:52.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f9870120640 1 -- start start 2024-04-23T18:35:52.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f9870120640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9868101fa0 0x7f9868198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:52.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f9870120640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9868103190 0x7f98681988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f9870120640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868103b50 0x7f986819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f9870120640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f986810dba0 con 0x7f9868103190 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f9870120640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f986810da20 con 0x7f9868101fa0 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f9870120640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f986810dd20 con 0x7f9868103b50 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f986d6bb640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9868103190 0x7f98681988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f986d6bb640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9868103190 0x7f98681988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60038/0 (socket says 172.21.15.132:60038) 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f986d6bb640 1 -- 172.21.15.132:0/1880790204 learned_addr learned my addr 172.21.15.132:0/1880790204 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f986d6bb640 1 -- 172.21.15.132:0/1880790204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868103b50 msgr2=0x7f986819f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f986d6bb640 1 --2- 172.21.15.132:0/1880790204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868103b50 0x7f986819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f986d6bb640 1 -- 172.21.15.132:0/1880790204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9868101fa0 msgr2=0x7f9868198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f986d6bb640 1 --2- 172.21.15.132:0/1880790204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9868101fa0 0x7f9868198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f986d6bb640 1 -- 172.21.15.132:0/1880790204 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986819ff80 con 0x7f9868103190 2024-04-23T18:35:52.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f986d6bb640 1 --2- 172.21.15.132:0/1880790204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9868103190 0x7f98681988a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f985800b4a0 tx=0x7f985800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:52.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f985effd640 1 -- 172.21.15.132:0/1880790204 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858014070 con 0x7f9868103190 2024-04-23T18:35:52.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f9870120640 1 -- 172.21.15.132:0/1880790204 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98681a0270 con 0x7f9868103190 2024-04-23T18:35:52.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f985effd640 1 -- 172.21.15.132:0/1880790204 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9858010ac0 con 0x7f9868103190 2024-04-23T18:35:52.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f985effd640 1 -- 172.21.15.132:0/1880790204 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858017c30 con 0x7f9868103190 2024-04-23T18:35:52.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.607+0000 7f9870120640 1 -- 172.21.15.132:0/1880790204 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98681a0840 con 0x7f9868103190 2024-04-23T18:35:52.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.608+0000 7f9870120640 1 -- 172.21.15.132:0/1880790204 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9868109380 con 0x7f9868103190 2024-04-23T18:35:52.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.610+0000 7f985effd640 1 -- 172.21.15.132:0/1880790204 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f9858017070 con 0x7f9868103190 2024-04-23T18:35:52.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.610+0000 7f985effd640 1 --2- 172.21.15.132:0/1880790204 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98540a3bb0 0x7f98540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:52.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.610+0000 7f986debc640 1 --2- 172.21.15.132:0/1880790204 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98540a3bb0 0x7f98540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:52.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.610+0000 7f985effd640 1 -- 172.21.15.132:0/1880790204 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f98580cfbe0 con 0x7f9868103190 2024-04-23T18:35:52.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.611+0000 7f986debc640 1 --2- 172.21.15.132:0/1880790204 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98540a3bb0 0x7f98540a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f98640096c0 tx=0x7f9864000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:52.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.614+0000 7f985effd640 1 -- 172.21.15.132:0/1880790204 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f985809c5b0 con 0x7f9868103190 2024-04-23T18:35:52.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.796+0000 7f9870120640 1 -- 172.21.15.132:0/1880790204 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f986802d870 con 0x7f9868103190 2024-04-23T18:35:52.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:52 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2079904825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:35:52.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:52 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/634104928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:35:52.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:52 smithi157 ceph-mon[32580]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:35:52.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:52 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1915297192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:35:52.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:52 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1085750503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:52.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.798+0000 7f985effd640 1 -- 172.21.15.132:0/1880790204 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22800 (secure 0 0 0) 0x7f985809beb0 con 0x7f9868103190 2024-04-23T18:35:52.790 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:52.791 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":88,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:34:55.248852+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:35:52.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.800+0000 7f9870120640 1 -- 172.21.15.132:0/1880790204 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98540a3bb0 msgr2=0x7f98540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.800+0000 7f9870120640 1 --2- 172.21.15.132:0/1880790204 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98540a3bb0 0x7f98540a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f98640096c0 tx=0x7f9864000bc0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.800+0000 7f9870120640 1 -- 172.21.15.132:0/1880790204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9868103190 msgr2=0x7f98681988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.800+0000 7f9870120640 1 --2- 172.21.15.132:0/1880790204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9868103190 0x7f98681988a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f985800b4a0 tx=0x7f985800b960 comp rx=0 tx=0).stop 2024-04-23T18:35:52.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.800+0000 7f9870120640 1 -- 172.21.15.132:0/1880790204 shutdown_connections 2024-04-23T18:35:52.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.800+0000 7f9870120640 1 --2- 172.21.15.132:0/1880790204 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f98540a3bb0 0x7f98540a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.800+0000 7f9870120640 1 --2- 172.21.15.132:0/1880790204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868103b50 0x7f986819f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.800+0000 7f9870120640 1 --2- 172.21.15.132:0/1880790204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9868103190 0x7f98681988a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.800+0000 7f9870120640 1 --2- 172.21.15.132:0/1880790204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9868101fa0 0x7f9868198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.800+0000 7f9870120640 1 -- 172.21.15.132:0/1880790204 >> 172.21.15.132:0/1880790204 conn(0x7f98680fd530 msgr2=0x7f9868106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:52.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.800+0000 7f9870120640 1 -- 172.21.15.132:0/1880790204 shutdown_connections 2024-04-23T18:35:52.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.800+0000 7f9870120640 1 -- 172.21.15.132:0/1880790204 wait complete. 2024-04-23T18:35:52.802 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:52 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2079904825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:35:52.802 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:52 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/634104928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:35:52.802 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:52 smithi132 ceph-mon[29178]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:35:52.802 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:52 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1915297192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:35:52.802 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:52 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1085750503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:52.804 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:29:20.132451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:33:58.407521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:33:59.518206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:35:52.804 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:35:52.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 -- 172.21.15.132:0/162809347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59941041b0 msgr2=0x7f59941045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 --2- 172.21.15.132:0/162809347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59941041b0 0x7f59941045d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f598c009ca0 tx=0x7f598c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 -- 172.21.15.132:0/162809347 shutdown_connections 2024-04-23T18:35:52.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 --2- 172.21.15.132:0/162809347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59941053a0 0x7f5994105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 --2- 172.21.15.132:0/162809347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5994105d60 0x7f599410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 --2- 172.21.15.132:0/162809347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59941041b0 0x7f59941045d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 -- 172.21.15.132:0/162809347 >> 172.21.15.132:0/162809347 conn(0x7f59940ff700 msgr2=0x7f5994101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:52.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 -- 172.21.15.132:0/162809347 shutdown_connections 2024-04-23T18:35:52.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 -- 172.21.15.132:0/162809347 wait complete. 2024-04-23T18:35:52.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 Processor -- start 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 -- start start 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59941041b0 0x7f599419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59941053a0 0x7f599419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5994105d60 0x7f59941a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f599410fcf0 con 0x7f5994105d60 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f599410fb70 con 0x7f59941041b0 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.912+0000 7f599b0e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f599410fe70 con 0x7f59941053a0 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f599967e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5994105d60 0x7f59941a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f599967e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5994105d60 0x7f59941a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60050/0 (socket says 172.21.15.132:60050) 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f599967e640 1 -- 172.21.15.132:0/1112017408 learned_addr learned my addr 172.21.15.132:0/1112017408 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f5998e7d640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59941041b0 0x7f599419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f5993fff640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59941053a0 0x7f599419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f599967e640 1 -- 172.21.15.132:0/1112017408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59941053a0 msgr2=0x7f599419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f599967e640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59941053a0 0x7f599419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f599967e640 1 -- 172.21.15.132:0/1112017408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59941041b0 msgr2=0x7f599419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:52.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f599967e640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59941041b0 0x7f599419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:52.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f599967e640 1 -- 172.21.15.132:0/1112017408 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59941a1ff0 con 0x7f5994105d60 2024-04-23T18:35:52.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f599967e640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5994105d60 0x7f59941a1980 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f597c00c970 tx=0x7f597c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:52.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f5991ffb640 1 -- 172.21.15.132:0/1112017408 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c007bb0 con 0x7f5994105d60 2024-04-23T18:35:52.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f5991ffb640 1 -- 172.21.15.132:0/1112017408 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f597c007d50 con 0x7f5994105d60 2024-04-23T18:35:52.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f5991ffb640 1 -- 172.21.15.132:0/1112017408 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c018770 con 0x7f5994105d60 2024-04-23T18:35:52.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f599b0e1640 1 -- 172.21.15.132:0/1112017408 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59941a22e0 con 0x7f5994105d60 2024-04-23T18:35:52.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.913+0000 7f599b0e1640 1 -- 172.21.15.132:0/1112017408 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59941a27e0 con 0x7f5994105d60 2024-04-23T18:35:52.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.914+0000 7f599b0e1640 1 -- 172.21.15.132:0/1112017408 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f599410b5a0 con 0x7f5994105d60 2024-04-23T18:35:52.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.915+0000 7f5991ffb640 1 -- 172.21.15.132:0/1112017408 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f597c00f990 con 0x7f5994105d60 2024-04-23T18:35:52.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.916+0000 7f5991ffb640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f59800a3bb0 0x7f59800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:52.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.916+0000 7f5998e7d640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f59800a3bb0 0x7f59800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:52.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.916+0000 7f5991ffb640 1 -- 172.21.15.132:0/1112017408 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f597c0d05a0 con 0x7f5994105d60 2024-04-23T18:35:52.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.916+0000 7f5998e7d640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f59800a3bb0 0x7f59800a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f598c000f80 tx=0x7f598c0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:52.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:52.920+0000 7f5991ffb640 1 -- 172.21.15.132:0/1112017408 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f597c014030 con 0x7f5994105d60 2024-04-23T18:35:53.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:52 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2079904825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:35:53.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:52 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/634104928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:35:53.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:52 smithi148 ceph-mon[32311]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T18:35:53.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:52 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1915297192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:35:53.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:52 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1085750503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:53.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.097+0000 7f599b0e1640 1 -- 172.21.15.132:0/1112017408 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5994062b50 con 0x7f5994105d60 2024-04-23T18:35:53.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.097+0000 7f5991ffb640 1 -- 172.21.15.132:0/1112017408 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f597c09cf70 con 0x7f5994105d60 2024-04-23T18:35:53.090 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-04-23T18:35:53.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.099+0000 7f599b0e1640 1 -- 172.21.15.132:0/1112017408 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f59800a3bb0 msgr2=0x7f59800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.099+0000 7f599b0e1640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f59800a3bb0 0x7f59800a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f598c000f80 tx=0x7f598c0093f0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.099+0000 7f599b0e1640 1 -- 172.21.15.132:0/1112017408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5994105d60 msgr2=0x7f59941a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.099+0000 7f599b0e1640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5994105d60 0x7f59941a1980 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f597c00c970 tx=0x7f597c00ce30 comp rx=0 tx=0).stop 2024-04-23T18:35:53.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.099+0000 7f599b0e1640 1 -- 172.21.15.132:0/1112017408 shutdown_connections 2024-04-23T18:35:53.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.099+0000 7f599b0e1640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f59800a3bb0 0x7f59800a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.099+0000 7f599b0e1640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59941053a0 0x7f599419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.099+0000 7f599b0e1640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5994105d60 0x7f59941a1980 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.099+0000 7f599b0e1640 1 --2- 172.21.15.132:0/1112017408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59941041b0 0x7f599419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.099+0000 7f599b0e1640 1 -- 172.21.15.132:0/1112017408 >> 172.21.15.132:0/1112017408 conn(0x7f59940ff700 msgr2=0x7f5994108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:53.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.100+0000 7f599b0e1640 1 -- 172.21.15.132:0/1112017408 shutdown_connections 2024-04-23T18:35:53.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.100+0000 7f599b0e1640 1 -- 172.21.15.132:0/1112017408 wait complete. 2024-04-23T18:35:53.103 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:35:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 -- 172.21.15.132:0/3502511608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66b8101f80 msgr2=0x7f66b81023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 --2- 172.21.15.132:0/3502511608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66b8101f80 0x7f66b81023a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009900 tx=0x7f66ac02a410 comp rx=0 tx=0).stop 2024-04-23T18:35:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 -- 172.21.15.132:0/3502511608 shutdown_connections 2024-04-23T18:35:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 --2- 172.21.15.132:0/3502511608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66b8103170 0x7f66b81035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 --2- 172.21.15.132:0/3502511608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66b8103b30 0x7f66b810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 --2- 172.21.15.132:0/3502511608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66b8101f80 0x7f66b81023a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 -- 172.21.15.132:0/3502511608 >> 172.21.15.132:0/3502511608 conn(0x7f66b80fd510 msgr2=0x7f66b80ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 -- 172.21.15.132:0/3502511608 shutdown_connections 2024-04-23T18:35:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 -- 172.21.15.132:0/3502511608 wait complete. 2024-04-23T18:35:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 Processor -- start 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 -- start start 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66b8101f80 0x7f66b8198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66b8103170 0x7f66b8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66b8103b30 0x7f66b819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f66b810db20 con 0x7f66b8103b30 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f66b810d9a0 con 0x7f66b8103170 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bedad640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f66b810dca0 con 0x7f66b8101f80 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bd34a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66b8103b30 0x7f66b819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bd34a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66b8103b30 0x7f66b819f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60080/0 (socket says 172.21.15.132:60080) 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.212+0000 7f66bd34a640 1 -- 172.21.15.132:0/3791267850 learned_addr learned my addr 172.21.15.132:0/3791267850 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66bcb49640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66b8101f80 0x7f66b8198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66b7fff640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66b8103170 0x7f66b8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66bd34a640 1 -- 172.21.15.132:0/3791267850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66b8101f80 msgr2=0x7f66b8198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66bd34a640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66b8101f80 0x7f66b8198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66bd34a640 1 -- 172.21.15.132:0/3791267850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66b8103170 msgr2=0x7f66b8198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66bd34a640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66b8103170 0x7f66b8198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66bd34a640 1 -- 172.21.15.132:0/3791267850 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b819fe70 con 0x7f66b8103b30 2024-04-23T18:35:53.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66bd34a640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66b8103b30 0x7f66b819f800 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f66b000b970 tx=0x7f66b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:53.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66b5ffb640 1 -- 172.21.15.132:0/3791267850 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0019070 con 0x7f66b8103b30 2024-04-23T18:35:53.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66bedad640 1 -- 172.21.15.132:0/3791267850 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b81a0160 con 0x7f66b8103b30 2024-04-23T18:35:53.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66b5ffb640 1 -- 172.21.15.132:0/3791267850 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b0014440 con 0x7f66b8103b30 2024-04-23T18:35:53.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66b5ffb640 1 -- 172.21.15.132:0/3791267850 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00075d0 con 0x7f66b8103b30 2024-04-23T18:35:53.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.213+0000 7f66bedad640 1 -- 172.21.15.132:0/3791267850 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b81a06e0 con 0x7f66b8103b30 2024-04-23T18:35:53.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.214+0000 7f66bedad640 1 -- 172.21.15.132:0/3791267850 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b81092c0 con 0x7f66b8103b30 2024-04-23T18:35:53.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.215+0000 7f66b5ffb640 1 -- 172.21.15.132:0/3791267850 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f66b001c070 con 0x7f66b8103b30 2024-04-23T18:35:53.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.216+0000 7f66b5ffb640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f66a80a3bb0 0x7f66a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:53.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.216+0000 7f66bcb49640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f66a80a3bb0 0x7f66a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:53.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.216+0000 7f66b5ffb640 1 -- 172.21.15.132:0/3791267850 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f66b00cf9c0 con 0x7f66b8103b30 2024-04-23T18:35:53.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.216+0000 7f66bcb49640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f66a80a3bb0 0x7f66a80a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009230 tx=0x7f66ac0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:53.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.220+0000 7f66b5ffb640 1 -- 172.21.15.132:0/3791267850 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66b00d1e20 con 0x7f66b8103b30 2024-04-23T18:35:53.393 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.401+0000 7f66bedad640 1 -- 172.21.15.132:0/3791267850 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f66b8062b50 con 0x7f66b8103b30 2024-04-23T18:35:53.393 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.401+0000 7f66b5ffb640 1 -- 172.21.15.132:0/3791267850 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f66b009bd40 con 0x7f66b8103b30 2024-04-23T18:35:53.393 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-04-23T18:35:53.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.403+0000 7f66bedad640 1 -- 172.21.15.132:0/3791267850 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f66a80a3bb0 msgr2=0x7f66a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.403+0000 7f66bedad640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f66a80a3bb0 0x7f66a80a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009230 tx=0x7f66ac0314a0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.403+0000 7f66bedad640 1 -- 172.21.15.132:0/3791267850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66b8103b30 msgr2=0x7f66b819f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.403+0000 7f66bedad640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66b8103b30 0x7f66b819f800 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f66b000b970 tx=0x7f66b000be30 comp rx=0 tx=0).stop 2024-04-23T18:35:53.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.403+0000 7f66bedad640 1 -- 172.21.15.132:0/3791267850 shutdown_connections 2024-04-23T18:35:53.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.403+0000 7f66bedad640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f66a80a3bb0 0x7f66a80a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.403+0000 7f66bedad640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66b8103b30 0x7f66b819f800 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.403+0000 7f66bedad640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66b8103170 0x7f66b8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.403+0000 7f66bedad640 1 --2- 172.21.15.132:0/3791267850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66b8101f80 0x7f66b8198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.403+0000 7f66bedad640 1 -- 172.21.15.132:0/3791267850 >> 172.21.15.132:0/3791267850 conn(0x7f66b80fd510 msgr2=0x7f66b8106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:53.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.404+0000 7f66bedad640 1 -- 172.21.15.132:0/3791267850 shutdown_connections 2024-04-23T18:35:53.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.404+0000 7f66bedad640 1 -- 172.21.15.132:0/3791267850 wait complete. 2024-04-23T18:35:53.407 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:35:53.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.515+0000 7f6ad6889640 1 -- 172.21.15.132:0/4218654866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad0101fe0 msgr2=0x7f6ad0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 --2- 172.21.15.132:0/4218654866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad0101fe0 0x7f6ad0102460 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009900 tx=0x7f6ac402a410 comp rx=0 tx=0).stop 2024-04-23T18:35:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 -- 172.21.15.132:0/4218654866 shutdown_connections 2024-04-23T18:35:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 --2- 172.21.15.132:0/4218654866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad01029a0 0x7f6ad01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 --2- 172.21.15.132:0/4218654866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad0101fe0 0x7f6ad0102460 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 --2- 172.21.15.132:0/4218654866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad0100e90 0x7f6ad01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 -- 172.21.15.132:0/4218654866 >> 172.21.15.132:0/4218654866 conn(0x7f6ad00fc400 msgr2=0x7f6ad00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 -- 172.21.15.132:0/4218654866 shutdown_connections 2024-04-23T18:35:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 -- 172.21.15.132:0/4218654866 wait complete. 2024-04-23T18:35:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 Processor -- start 2024-04-23T18:35:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 -- start start 2024-04-23T18:35:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad0100e90 0x7f6ad0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad0101fe0 0x7f6ad0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad01029a0 0x7f6ad019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad010c9e0 con 0x7f6ad0100e90 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad010c860 con 0x7f6ad01029a0 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.516+0000 7f6ad6889640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad010cb60 con 0x7f6ad0101fe0 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad0100e90 0x7f6ad0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad0100e90 0x7f6ad0198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60096/0 (socket says 172.21.15.132:60096) 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acffff640 1 -- 172.21.15.132:0/3637376288 learned_addr learned my addr 172.21.15.132:0/3637376288 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6ad4e26640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad01029a0 0x7f6ad019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acf7fe640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad0101fe0 0x7f6ad0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acffff640 1 -- 172.21.15.132:0/3637376288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad0101fe0 msgr2=0x7f6ad0198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acffff640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad0101fe0 0x7f6ad0198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acffff640 1 -- 172.21.15.132:0/3637376288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad01029a0 msgr2=0x7f6ad019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acffff640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad01029a0 0x7f6ad019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acffff640 1 -- 172.21.15.132:0/3637376288 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad019fe50 con 0x7f6ad0100e90 2024-04-23T18:35:53.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6ad4e26640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad01029a0 0x7f6ad019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:53.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acf7fe640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad0101fe0 0x7f6ad0198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:53.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acffff640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad0100e90 0x7f6ad0198230 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e9f0 tx=0x7f6ac000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:53.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acd7fa640 1 -- 172.21.15.132:0/3637376288 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac000cd80 con 0x7f6ad0100e90 2024-04-23T18:35:53.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acd7fa640 1 -- 172.21.15.132:0/3637376288 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac00108e0 con 0x7f6ad0100e90 2024-04-23T18:35:53.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6ad6889640 1 -- 172.21.15.132:0/3637376288 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad01a0140 con 0x7f6ad0100e90 2024-04-23T18:35:53.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6acd7fa640 1 -- 172.21.15.132:0/3637376288 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0018980 con 0x7f6ad0100e90 2024-04-23T18:35:53.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.517+0000 7f6ad6889640 1 -- 172.21.15.132:0/3637376288 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad01a0690 con 0x7f6ad0100e90 2024-04-23T18:35:53.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.518+0000 7f6ad6889640 1 -- 172.21.15.132:0/3637376288 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad01a08f0 con 0x7f6ad0100e90 2024-04-23T18:35:53.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.519+0000 7f6acd7fa640 1 -- 172.21.15.132:0/3637376288 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f6ac0010430 con 0x7f6ad0100e90 2024-04-23T18:35:53.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.520+0000 7f6acd7fa640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6ab80a3bb0 0x7f6ab80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:53.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.520+0000 7f6acf7fe640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6ab80a3bb0 0x7f6ab80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:53.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.520+0000 7f6acd7fa640 1 -- 172.21.15.132:0/3637376288 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6ac00d0d40 con 0x7f6ad0100e90 2024-04-23T18:35:53.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.520+0000 7f6acf7fe640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6ab80a3bb0 0x7f6ab80a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4006010 tx=0x7f6ac4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:53.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.523+0000 7f6acd7fa640 1 -- 172.21.15.132:0/3637376288 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ac009d710 con 0x7f6ad0100e90 2024-04-23T18:35:53.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.703+0000 7f6ad6889640 1 -- 172.21.15.132:0/3637376288 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6ad01a0ac0 con 0x7f6ad0100e90 2024-04-23T18:35:53.696 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.703+0000 7f6acd7fa640 1 -- 172.21.15.132:0/3637376288 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f6ac009d010 con 0x7f6ad0100e90 2024-04-23T18:35:53.696 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-04-23T18:35:53.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.705+0000 7f6ad6889640 1 -- 172.21.15.132:0/3637376288 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6ab80a3bb0 msgr2=0x7f6ab80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.705+0000 7f6ad6889640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6ab80a3bb0 0x7f6ab80a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4006010 tx=0x7f6ac4000bc0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.705+0000 7f6ad6889640 1 -- 172.21.15.132:0/3637376288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad0100e90 msgr2=0x7f6ad0198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.705+0000 7f6ad6889640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad0100e90 0x7f6ad0198230 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e9f0 tx=0x7f6ac000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.706+0000 7f6ad6889640 1 -- 172.21.15.132:0/3637376288 shutdown_connections 2024-04-23T18:35:53.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.706+0000 7f6ad6889640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f6ab80a3bb0 0x7f6ab80a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.706+0000 7f6ad6889640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad0101fe0 0x7f6ad0198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.706+0000 7f6ad6889640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad01029a0 0x7f6ad019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.706+0000 7f6ad6889640 1 --2- 172.21.15.132:0/3637376288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad0100e90 0x7f6ad0198230 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.706+0000 7f6ad6889640 1 -- 172.21.15.132:0/3637376288 >> 172.21.15.132:0/3637376288 conn(0x7f6ad00fc400 msgr2=0x7f6ad0069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:53.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.706+0000 7f6ad6889640 1 -- 172.21.15.132:0/3637376288 shutdown_connections 2024-04-23T18:35:53.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.706+0000 7f6ad6889640 1 -- 172.21.15.132:0/3637376288 wait complete. 2024-04-23T18:35:53.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:53 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1880790204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:53.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:53 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1112017408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:35:53.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:53 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3791267850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:35:53.709 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:35:53.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.815+0000 7fe3094e8640 1 -- 172.21.15.132:0/337768157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe304101ff0 msgr2=0x7fe304102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.815+0000 7fe3094e8640 1 --2- 172.21.15.132:0/337768157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe304101ff0 0x7fe304102470 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009900 tx=0x7fe2f802a410 comp rx=0 tx=0).stop 2024-04-23T18:35:53.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.815+0000 7fe3094e8640 1 -- 172.21.15.132:0/337768157 shutdown_connections 2024-04-23T18:35:53.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.815+0000 7fe3094e8640 1 --2- 172.21.15.132:0/337768157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3041029b0 0x7fe3041092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.815+0000 7fe3094e8640 1 --2- 172.21.15.132:0/337768157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe304101ff0 0x7fe304102470 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.815+0000 7fe3094e8640 1 --2- 172.21.15.132:0/337768157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe304100ea0 0x7fe3041012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.815+0000 7fe3094e8640 1 -- 172.21.15.132:0/337768157 >> 172.21.15.132:0/337768157 conn(0x7fe3040fc430 msgr2=0x7fe3040fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.815+0000 7fe3094e8640 1 -- 172.21.15.132:0/337768157 shutdown_connections 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.815+0000 7fe3094e8640 1 -- 172.21.15.132:0/337768157 wait complete. 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.815+0000 7fe3094e8640 1 Processor -- start 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3094e8640 1 -- start start 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3094e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe304100ea0 0x7fe30406b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3094e8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe304101ff0 0x7fe30419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3094e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3041029b0 0x7fe30419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3094e8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe30410c9e0 con 0x7fe304101ff0 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3094e8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe30410c860 con 0x7fe304100ea0 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3094e8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe30410cb60 con 0x7fe3041029b0 2024-04-23T18:35:53.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3027fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe304101ff0 0x7fe30419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3027fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe304101ff0 0x7fe30419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60104/0 (socket says 172.21.15.132:60104) 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3027fc640 1 -- 172.21.15.132:0/1258418821 learned_addr learned my addr 172.21.15.132:0/1258418821 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3027fc640 1 -- 172.21.15.132:0/1258418821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3041029b0 msgr2=0x7fe30419f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3027fc640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3041029b0 0x7fe30419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3027fc640 1 -- 172.21.15.132:0/1258418821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe304100ea0 msgr2=0x7fe30406b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe302ffd640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe304100ea0 0x7fe30406b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3027fc640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe304100ea0 0x7fe30406b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3027fc640 1 -- 172.21.15.132:0/1258418821 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30419fdb0 con 0x7fe304101ff0 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe3027fc640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe304101ff0 0x7fe30419ceb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8006010 tx=0x7fe2f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.816+0000 7fe2ebfff640 1 -- 172.21.15.132:0/1258418821 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f803f030 con 0x7fe304101ff0 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.817+0000 7fe2ebfff640 1 -- 172.21.15.132:0/1258418821 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2f80318f0 con 0x7fe304101ff0 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.817+0000 7fe3094e8640 1 -- 172.21.15.132:0/1258418821 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3041a0040 con 0x7fe304101ff0 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.817+0000 7fe2ebfff640 1 -- 172.21.15.132:0/1258418821 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8035070 con 0x7fe304101ff0 2024-04-23T18:35:53.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.817+0000 7fe3094e8640 1 -- 172.21.15.132:0/1258418821 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3041a0530 con 0x7fe304101ff0 2024-04-23T18:35:53.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.818+0000 7fe3094e8640 1 -- 172.21.15.132:0/1258418821 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3041081d0 con 0x7fe304101ff0 2024-04-23T18:35:53.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.819+0000 7fe2ebfff640 1 -- 172.21.15.132:0/1258418821 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fe2f8042030 con 0x7fe304101ff0 2024-04-23T18:35:53.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.819+0000 7fe2ebfff640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe2ec0a3bb0 0x7fe2ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:53.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.819+0000 7fe302ffd640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe2ec0a3bb0 0x7fe2ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:53.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.819+0000 7fe2ebfff640 1 -- 172.21.15.132:0/1258418821 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe2f80f0a20 con 0x7fe304101ff0 2024-04-23T18:35:53.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.820+0000 7fe302ffd640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe2ec0a3bb0 0x7fe2ec0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009aa0 tx=0x7fe2f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:53.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:53.824+0000 7fe2ebfff640 1 -- 172.21.15.132:0/1258418821 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe2f80bd3f0 con 0x7fe304101ff0 2024-04-23T18:35:54.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:53 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1880790204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:54.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:53 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1112017408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:35:54.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:53 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3791267850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:35:54.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.037+0000 7fe3094e8640 1 -- 172.21.15.132:0/1258418821 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe304062bf0 con 0x7fe304101ff0 2024-04-23T18:35:54.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.038+0000 7fe2ebfff640 1 -- 172.21.15.132:0/1258418821 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11946 (secure 0 0 0) 0x7fe2f80bccf0 con 0x7fe304101ff0 2024-04-23T18:35:54.030 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:54.030 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:34:07.536766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24394,24425,24460]},"id":1}]} 2024-04-23T18:35:54.030 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 25 2024-04-23T18:35:54.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.040+0000 7fe3094e8640 1 -- 172.21.15.132:0/1258418821 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe2ec0a3bb0 msgr2=0x7fe2ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.040+0000 7fe3094e8640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe2ec0a3bb0 0x7fe2ec0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009aa0 tx=0x7fe2f4005f90 comp rx=0 tx=0).stop 2024-04-23T18:35:54.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.040+0000 7fe3094e8640 1 -- 172.21.15.132:0/1258418821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe304101ff0 msgr2=0x7fe30419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.040+0000 7fe3094e8640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe304101ff0 0x7fe30419ceb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8006010 tx=0x7fe2f80042f0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.040+0000 7fe3094e8640 1 -- 172.21.15.132:0/1258418821 shutdown_connections 2024-04-23T18:35:54.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.040+0000 7fe3094e8640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe2ec0a3bb0 0x7fe2ec0a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.040+0000 7fe3094e8640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3041029b0 0x7fe30419f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.040+0000 7fe3094e8640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe304101ff0 0x7fe30419ceb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.040+0000 7fe3094e8640 1 --2- 172.21.15.132:0/1258418821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe304100ea0 0x7fe30406b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.040+0000 7fe3094e8640 1 -- 172.21.15.132:0/1258418821 >> 172.21.15.132:0/1258418821 conn(0x7fe3040fc430 msgr2=0x7fe3041053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.040+0000 7fe3094e8640 1 -- 172.21.15.132:0/1258418821 shutdown_connections 2024-04-23T18:35:54.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.040+0000 7fe3094e8640 1 -- 172.21.15.132:0/1258418821 wait complete. 2024-04-23T18:35:54.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:53 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1880790204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:54.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:53 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1112017408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:35:54.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:53 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3791267850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:35:54.043 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:35:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.150+0000 7fec9a660640 1 -- 172.21.15.132:0/1432501842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec94103b70 msgr2=0x7fec9410a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.150+0000 7fec9a660640 1 --2- 172.21.15.132:0/1432501842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec94103b70 0x7fec9410a400 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009ca0 tx=0x7fec8c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 -- 172.21.15.132:0/1432501842 shutdown_connections 2024-04-23T18:35:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 --2- 172.21.15.132:0/1432501842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec941031b0 0x7fec94103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 --2- 172.21.15.132:0/1432501842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec94103b70 0x7fec9410a400 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 --2- 172.21.15.132:0/1432501842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec94101fc0 0x7fec941023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 -- 172.21.15.132:0/1432501842 >> 172.21.15.132:0/1432501842 conn(0x7fec940fd530 msgr2=0x7fec940ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 -- 172.21.15.132:0/1432501842 shutdown_connections 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 -- 172.21.15.132:0/1432501842 wait complete. 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 Processor -- start 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 -- start start 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec94101fc0 0x7fec94198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec941031b0 0x7fec94198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec94103b70 0x7fec9419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fec9410db00 con 0x7fec941031b0 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fec9410d980 con 0x7fec94101fc0 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.151+0000 7fec9a660640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fec9410dc80 con 0x7fec94103b70 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec937fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec941031b0 0x7fec94198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec937fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec941031b0 0x7fec94198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60108/0 (socket says 172.21.15.132:60108) 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec937fe640 1 -- 172.21.15.132:0/769181272 learned_addr learned my addr 172.21.15.132:0/769181272 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec93fff640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec94101fc0 0x7fec94198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec98bfd640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec94103b70 0x7fec9419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec937fe640 1 -- 172.21.15.132:0/769181272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec94103b70 msgr2=0x7fec9419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec937fe640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec94103b70 0x7fec9419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec937fe640 1 -- 172.21.15.132:0/769181272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec94101fc0 msgr2=0x7fec94198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec937fe640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec94101fc0 0x7fec94198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec937fe640 1 -- 172.21.15.132:0/769181272 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9419fd40 con 0x7fec941031b0 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec98bfd640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec94103b70 0x7fec9419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec937fe640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec941031b0 0x7fec94198660 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fec8800c970 tx=0x7fec8800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec917fa640 1 -- 172.21.15.132:0/769181272 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88007bb0 con 0x7fec941031b0 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec9a660640 1 -- 172.21.15.132:0/769181272 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9419ffd0 con 0x7fec941031b0 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec917fa640 1 -- 172.21.15.132:0/769181272 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec88007d50 con 0x7fec941031b0 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec917fa640 1 -- 172.21.15.132:0/769181272 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8800f450 con 0x7fec941031b0 2024-04-23T18:35:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.152+0000 7fec9a660640 1 -- 172.21.15.132:0/769181272 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec941a04a0 con 0x7fec941031b0 2024-04-23T18:35:54.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.153+0000 7fec9a660640 1 -- 172.21.15.132:0/769181272 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec94109280 con 0x7fec941031b0 2024-04-23T18:35:54.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.154+0000 7fec917fa640 1 -- 172.21.15.132:0/769181272 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fec88004f20 con 0x7fec941031b0 2024-04-23T18:35:54.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.155+0000 7fec917fa640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fec7c0a3bb0 0x7fec7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.155+0000 7fec93fff640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fec7c0a3bb0 0x7fec7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.155+0000 7fec917fa640 1 -- 172.21.15.132:0/769181272 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fec880d0160 con 0x7fec941031b0 2024-04-23T18:35:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.155+0000 7fec93fff640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fec7c0a3bb0 0x7fec7c0a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec94103010 tx=0x7fec84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.159+0000 7fec917fa640 1 -- 172.21.15.132:0/769181272 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec88014030 con 0x7fec941031b0 2024-04-23T18:35:54.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.339+0000 7fec9a660640 1 -- 172.21.15.132:0/769181272 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec94062b50 con 0x7fec941031b0 2024-04-23T18:35:54.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.340+0000 7fec917fa640 1 -- 172.21.15.132:0/769181272 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22800 (secure 0 0 0) 0x7fec8809cb30 con 0x7fec941031b0 2024-04-23T18:35:54.333 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:35:54.333 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":88,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:34:55.248852+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:35:54.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.342+0000 7fec9a660640 1 -- 172.21.15.132:0/769181272 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fec7c0a3bb0 msgr2=0x7fec7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.342+0000 7fec9a660640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fec7c0a3bb0 0x7fec7c0a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec94103010 tx=0x7fec84005f90 comp rx=0 tx=0).stop 2024-04-23T18:35:54.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.342+0000 7fec9a660640 1 -- 172.21.15.132:0/769181272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec941031b0 msgr2=0x7fec94198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.342+0000 7fec9a660640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec941031b0 0x7fec94198660 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fec8800c970 tx=0x7fec8800ce30 comp rx=0 tx=0).stop 2024-04-23T18:35:54.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.342+0000 7fec9a660640 1 -- 172.21.15.132:0/769181272 shutdown_connections 2024-04-23T18:35:54.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.342+0000 7fec9a660640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fec7c0a3bb0 0x7fec7c0a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.342+0000 7fec9a660640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec94103b70 0x7fec9419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.342+0000 7fec9a660640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec941031b0 0x7fec94198660 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.342+0000 7fec9a660640 1 --2- 172.21.15.132:0/769181272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec94101fc0 0x7fec94198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.342+0000 7fec9a660640 1 -- 172.21.15.132:0/769181272 >> 172.21.15.132:0/769181272 conn(0x7fec940fd530 msgr2=0x7fec94106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.342+0000 7fec9a660640 1 -- 172.21.15.132:0/769181272 shutdown_connections 2024-04-23T18:35:54.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.342+0000 7fec9a660640 1 -- 172.21.15.132:0/769181272 wait complete. 2024-04-23T18:35:54.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T18:34:00.571339+0000', 'modified': '2024-04-23T18:34:07.536766+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24515, 'mds_1': 24491, 'mds_2': 24460, 'mds_3': 24425, 'mds_4': 24394}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6839/579251026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 579251026}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 579251026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6841/826931131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6840', 'nonce': 826931131}, {'type': 'v1', 'addr': '172.21.15.132:6841', 'nonce': 826931131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24394': {'gid': 24394, 'name': 'l', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.157:6833/1034092547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 1034092547}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 1034092547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.148:6833/3556073608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 3556073608}, {'type': 'v1', 'addr': '172.21.15.148:6833', 'nonce': 3556073608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/4121037781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 4121037781}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 4121037781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6837/2670322328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 2670322328}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 2670322328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/397710093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 397710093}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 397710093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.157:6839/1528717147', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 1528717147}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 1528717147}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.148:6837/2626530887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 2626530887}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 2626530887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6839/4115412726', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 4115412726}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 4115412726}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24515, 'qdb_cluster': [24491, 24515, 24394, 24425, 24460]} 2024-04-23T18:35:54.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-23T18:35:54.348 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-23T18:35:54.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.455+0000 7f571ef07640 1 -- 172.21.15.132:0/2417181782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57181053c0 msgr2=0x7f57181077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.455+0000 7f571ef07640 1 --2- 172.21.15.132:0/2417181782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57181053c0 0x7f57181077b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5704009900 tx=0x7f570402a410 comp rx=0 tx=0).stop 2024-04-23T18:35:54.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.455+0000 7f571ef07640 1 -- 172.21.15.132:0/2417181782 shutdown_connections 2024-04-23T18:35:54.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.455+0000 7f571ef07640 1 --2- 172.21.15.132:0/2417181782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5718107cf0 0x7f571810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.455+0000 7f571ef07640 1 --2- 172.21.15.132:0/2417181782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57181053c0 0x7f57181077b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.455+0000 7f571ef07640 1 --2- 172.21.15.132:0/2417181782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5718102a90 0x7f5718104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.455+0000 7f571ef07640 1 -- 172.21.15.132:0/2417181782 >> 172.21.15.132:0/2417181782 conn(0x7f57180fc440 msgr2=0x7f57180fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.455+0000 7f571ef07640 1 -- 172.21.15.132:0/2417181782 shutdown_connections 2024-04-23T18:35:54.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.455+0000 7f571ef07640 1 -- 172.21.15.132:0/2417181782 wait complete. 2024-04-23T18:35:54.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571ef07640 1 Processor -- start 2024-04-23T18:35:54.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571ef07640 1 -- start start 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571ef07640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5718102a90 0x7f57181981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571ef07640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57181053c0 0x7f57181986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571ef07640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5718107cf0 0x7f571819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571ef07640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f571810db30 con 0x7f5718107cf0 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571ef07640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f571810d9b0 con 0x7f5718102a90 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571ef07640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f571810dcb0 con 0x7f57181053c0 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571d4a4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5718107cf0 0x7f571819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571d4a4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5718107cf0 0x7f571819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60124/0 (socket says 172.21.15.132:60124) 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571d4a4640 1 -- 172.21.15.132:0/3465179751 learned_addr learned my addr 172.21.15.132:0/3465179751 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571cca3640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5718102a90 0x7f57181981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f5717fff640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57181053c0 0x7f57181986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571d4a4640 1 -- 172.21.15.132:0/3465179751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57181053c0 msgr2=0x7f57181986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571d4a4640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57181053c0 0x7f57181986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571d4a4640 1 -- 172.21.15.132:0/3465179751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5718102a90 msgr2=0x7f57181981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571d4a4640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5718102a90 0x7f57181981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571d4a4640 1 -- 172.21.15.132:0/3465179751 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571819fdc0 con 0x7f5718107cf0 2024-04-23T18:35:54.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f5717fff640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57181053c0 0x7f57181986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:54.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571cca3640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5718102a90 0x7f57181981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:54.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571d4a4640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5718107cf0 0x7f571819f750 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f571000b970 tx=0x7f571000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f5715ffb640 1 -- 172.21.15.132:0/3465179751 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710019070 con 0x7f5718107cf0 2024-04-23T18:35:54.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f5715ffb640 1 -- 172.21.15.132:0/3465179751 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5710014440 con 0x7f5718107cf0 2024-04-23T18:35:54.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f571ef07640 1 -- 172.21.15.132:0/3465179751 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57181a00b0 con 0x7f5718107cf0 2024-04-23T18:35:54.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.456+0000 7f5715ffb640 1 -- 172.21.15.132:0/3465179751 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57100075d0 con 0x7f5718107cf0 2024-04-23T18:35:54.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.457+0000 7f571ef07640 1 -- 172.21.15.132:0/3465179751 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57181a0680 con 0x7f5718107cf0 2024-04-23T18:35:54.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.458+0000 7f5715ffb640 1 -- 172.21.15.132:0/3465179751 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f571001c070 con 0x7f5718107cf0 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f5721c16740 1 Processor -- start 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f5721c16740 1 -- start start 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f5715ffb640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f57080a3c30 0x7f57080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f5721c16740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468628fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f5721c16740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x55946862a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f5721c16740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x559468630fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f5721c16740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55946858dc50 con 0x55946862a710 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f5721c16740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55946852a830 con 0x559468628b80 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f5721c16740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559468589c10 con 0x559468629d50 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f5717fff640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f57080a3c30 0x7f57080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f571d4a4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x559468630fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f571cca3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468628fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f571d4a4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x559468630fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60132/0 (socket says 172.21.15.132:60132) 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f571d4a4640 1 -- 172.21.15.132:0/4027048192 learned_addr learned my addr 172.21.15.132:0/4027048192 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f5715ffb640 1 -- 172.21.15.132:0/3465179751 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f57100cf9e0 con 0x7f5718107cf0 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f571d4a4640 1 -- 172.21.15.132:0/4027048192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 msgr2=0x55946862a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f5717fff640 1 --2- 172.21.15.132:0/4027048192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x55946862a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f571d4a4640 1 --2- 172.21.15.132:0/4027048192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x55946862a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f571d4a4640 1 -- 172.21.15.132:0/4027048192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 msgr2=0x559468628fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f571d4a4640 1 --2- 172.21.15.132:0/4027048192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468628fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.459+0000 7f571d4a4640 1 -- 172.21.15.132:0/4027048192 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594682a02d0 con 0x55946862a710 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5717fff640 1 --2- 172.21.15.132:0/4027048192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x55946862a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f571cca3640 1 --2- 172.21.15.132:0/4027048192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468628fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5717fff640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f57080a3c30 0x7f57080a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5704006010 tx=0x7f57040095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f571d4a4640 1 --2- 172.21.15.132:0/4027048192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x559468630fa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5710098110 tx=0x7f5710004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fce35a9b713f09b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5702ffd640 1 -- 172.21.15.132:0/4027048192 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57100a4920 con 0x55946862a710 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5702ffd640 1 -- 172.21.15.132:0/4027048192 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5710085240 con 0x55946862a710 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5702ffd640 1 -- 172.21.15.132:0/4027048192 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710098320 con 0x55946862a710 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5721c16740 1 -- 172.21.15.132:0/4027048192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 msgr2=0x559468630fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5721c16740 1 --2- 172.21.15.132:0/4027048192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x559468630fa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5710098110 tx=0x7f5710004b50 comp rx=0 tx=0).stop 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5721c16740 1 -- 172.21.15.132:0/4027048192 shutdown_connections 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5721c16740 1 --2- 172.21.15.132:0/4027048192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x55946862a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5721c16740 1 --2- 172.21.15.132:0/4027048192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x559468630fa0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5721c16740 1 --2- 172.21.15.132:0/4027048192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468628fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5721c16740 1 -- 172.21.15.132:0/4027048192 >> 172.21.15.132:0/4027048192 conn(0x559468624070 msgr2=0x559468624470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5721c16740 1 -- 172.21.15.132:0/4027048192 shutdown_connections 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.460+0000 7f5721c16740 1 -- 172.21.15.132:0/4027048192 wait complete. 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f5721c16740 1 -- start start 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f5721c16740 1 Processor -- start 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f5721c16740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468633000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f5721c16740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x559468633540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f5721c16740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x55946863a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f5721c16740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x559468631720 con 0x55946862a710 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f5721c16740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559468528470 con 0x559468628b80 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f5721c16740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5594686318a0 con 0x559468629d50 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f5721c16740 10 client.0 _ll_register_callbacks cb 0x55946854df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f571d4a4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x55946863a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f571d4a4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x55946863a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60136/0 (socket says 172.21.15.132:60136) 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f571d4a4640 1 -- 172.21.15.132:0/3391325734 learned_addr learned my addr 172.21.15.132:0/3391325734 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f571cca3640 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468633000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.461+0000 7f5717fff640 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x559468633540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f571d4a4640 1 -- 172.21.15.132:0/3391325734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 msgr2=0x559468633540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f571d4a4640 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x559468633540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f571d4a4640 1 -- 172.21.15.132:0/3391325734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 msgr2=0x559468633000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f571d4a4640 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468633000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f571d4a4640 1 -- 172.21.15.132:0/3391325734 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946836a6b0 con 0x55946862a710 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f571d4a4640 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x55946863a580 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f571009dff0 tx=0x7f57100a01a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882c5b178f71b284 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f56f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f56f7fff640 1 -- 172.21.15.132:0/3391325734 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57100a4920 con 0x55946862a710 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f56f7fff640 1 -- 172.21.15.132:0/3391325734 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5710050110 con 0x55946862a710 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f5721c16740 1 -- 172.21.15.132:0/3391325734 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5710085240 con 0x55946862a710 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f56f7fff640 1 -- 172.21.15.132:0/3391325734 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f57100b5020 con 0x55946862a710 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f5721c16740 10 client.15264 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f5721c16740 1 -- 172.21.15.132:0/3391325734 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55946863d860 con 0x55946862a710 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f56f7fff640 1 -- 172.21.15.132:0/3391325734 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5710085240 con 0x55946862a710 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f56f7fff640 1 -- 172.21.15.132:0/3391325734 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f57100a4490 con 0x55946862a710 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f5721c16740 10 client.15264 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f5721c16740 10 client.15264 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f5721c16740 20 client.15264 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f5721c16740 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x55946863f950 0x55946863ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f5721c16740 4 client.15264 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.462+0000 7f5721c16740 1 -- 172.21.15.132:0/3391325734 --> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55946859b670 con 0x55946863f950 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.463+0000 7f571cca3640 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x55946863f950 0x55946863ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.463+0000 7f571cca3640 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x55946863f950 0x55946863ebb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=493b39e1f9beb30b server_cookie=f28d6f537cddb392 in_seq=0 out_seq=0 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.463+0000 7f56f7fff640 10 client.15264 ms_handle_connect on v2:172.21.15.157:6832/1034092547 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f56f7fff640 1 -- 172.21.15.132:0/3391325734 <== mds.4 v2:172.21.15.157:6832/1034092547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55946859b670 con 0x55946863f950 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f56f7fff640 10 client.15264 handle_command_reply: tid=0 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f5721c16740 1 client.15264 shutdown 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f5721c16740 1 -- 172.21.15.132:0/3391325734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 msgr2=0x55946863a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f5721c16740 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x55946863a580 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f571009dff0 tx=0x7f57100a01a0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f5721c16740 1 -- 172.21.15.132:0/3391325734 shutdown_connections 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f5721c16740 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x55946863f950 0x55946863ebb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f5721c16740 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x559468633540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f5721c16740 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x55946863a580 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f56f7fff640 0 client.15264 ms_handle_reset on v2:172.21.15.157:6832/1034092547 2024-04-23T18:35:54.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f5721c16740 1 --2- 172.21.15.132:0/3391325734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468633000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f5721c16740 1 -- 172.21.15.132:0/3391325734 >> 172.21.15.132:0/3391325734 conn(0x559468624070 msgr2=0x5594686231d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f5721c16740 1 -- 172.21.15.132:0/3391325734 shutdown_connections 2024-04-23T18:35:54.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f5721c16740 1 -- 172.21.15.132:0/3391325734 wait complete. 2024-04-23T18:35:54.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.464+0000 7f5721c16740 20 client.15264 trim_cache size 0 max 16384 2024-04-23T18:35:54.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.482+0000 7f5721c16740 1 Processor -- start 2024-04-23T18:35:54.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.482+0000 7f5721c16740 1 -- start start 2024-04-23T18:35:54.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.482+0000 7f5721c16740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468628b80 0x559468630e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.482+0000 7f5721c16740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559468629d50 0x55946863a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.482+0000 7f5721c16740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55946862a710 0x559468632fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.482+0000 7f5721c16740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55946863f100 con 0x559468629d50 2024-04-23T18:35:54.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.482+0000 7f5721c16740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55946863f7d0 con 0x55946862a710 2024-04-23T18:35:54.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.482+0000 7f5721c16740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55946863fea0 con 0x559468628b80 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.482+0000 7f5717fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559468629d50 0x55946863a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.482+0000 7f5717fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559468629d50 0x55946863a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60148/0 (socket says 172.21.15.132:60148) 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.482+0000 7f5717fff640 1 -- 172.21.15.132:0/1375595531 learned_addr learned my addr 172.21.15.132:0/1375595531 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.483+0000 7f571cca3640 1 --2- 172.21.15.132:0/1375595531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468628b80 0x559468630e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.483+0000 7f571d4a4640 1 --2- 172.21.15.132:0/1375595531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55946862a710 0x559468632fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.483+0000 7f5717fff640 1 -- 172.21.15.132:0/1375595531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468628b80 msgr2=0x559468630e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.483+0000 7f5717fff640 1 --2- 172.21.15.132:0/1375595531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468628b80 0x559468630e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.483+0000 7f5717fff640 1 -- 172.21.15.132:0/1375595531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55946862a710 msgr2=0x559468632fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.483+0000 7f5717fff640 1 --2- 172.21.15.132:0/1375595531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55946862a710 0x559468632fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.483+0000 7f5717fff640 1 -- 172.21.15.132:0/1375595531 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559468633ea0 con 0x559468629d50 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.483+0000 7f571d4a4640 1 --2- 172.21.15.132:0/1375595531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55946862a710 0x559468632fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.483+0000 7f5717fff640 1 --2- 172.21.15.132:0/1375595531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559468629d50 0x55946863a540 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5704033000 tx=0x7f5704000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e749996184de0e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.483+0000 7f57027fc640 1 -- 172.21.15.132:0/1375595531 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570404f070 con 0x559468629d50 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.483+0000 7f57027fc640 1 -- 172.21.15.132:0/1375595531 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5704046040 con 0x559468629d50 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.483+0000 7f57027fc640 1 -- 172.21.15.132:0/1375595531 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704045040 con 0x559468629d50 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 -- 172.21.15.132:0/1375595531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559468629d50 msgr2=0x55946863a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 --2- 172.21.15.132:0/1375595531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559468629d50 0x55946863a540 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5704033000 tx=0x7f5704000bc0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 -- 172.21.15.132:0/1375595531 shutdown_connections 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 --2- 172.21.15.132:0/1375595531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55946862a710 0x559468632fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 --2- 172.21.15.132:0/1375595531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559468629d50 0x55946863a540 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 --2- 172.21.15.132:0/1375595531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468628b80 0x559468630e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 -- 172.21.15.132:0/1375595531 >> 172.21.15.132:0/1375595531 conn(0x559468624070 msgr2=0x55946861f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 -- 172.21.15.132:0/1375595531 shutdown_connections 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 -- 172.21.15.132:0/1375595531 wait complete. 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 -- start start 2024-04-23T18:35:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 Processor -- start 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468636000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x559468636540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x559468639560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x559468640c40 con 0x55946862a710 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559468640570 con 0x559468628b80 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559468641310 con 0x559468629d50 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.484+0000 7f5721c16740 10 client.0 _ll_register_callbacks cb 0x7f570c008390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f571d4a4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x559468639560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f571d4a4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x559468639560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60150/0 (socket says 172.21.15.132:60150) 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f571d4a4640 1 -- 172.21.15.132:0/3507433952 learned_addr learned my addr 172.21.15.132:0/3507433952 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f571cca3640 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468636000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f5717fff640 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x559468636540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f571d4a4640 1 -- 172.21.15.132:0/3507433952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 msgr2=0x559468636540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f571d4a4640 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x559468636540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f571d4a4640 1 -- 172.21.15.132:0/3507433952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 msgr2=0x559468636000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f571d4a4640 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468636000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f571d4a4640 1 -- 172.21.15.132:0/3507433952 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559468639b50 con 0x55946862a710 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f571cca3640 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468636000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f571d4a4640 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x559468639560 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f571009bc40 tx=0x7f57100cd520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caf29d9ed0d0617 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f56f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f56f7fff640 1 -- 172.21.15.132:0/3507433952 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710097f50 con 0x55946862a710 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f56f7fff640 1 -- 172.21.15.132:0/3507433952 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f57100a4490 con 0x55946862a710 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f5721c16740 1 -- 172.21.15.132:0/3507433952 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5704046040 con 0x55946862a710 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f56f7fff640 1 -- 172.21.15.132:0/3507433952 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f57100a4690 con 0x55946862a710 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f5721c16740 10 client.15270 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f5721c16740 1 -- 172.21.15.132:0/3507433952 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5594686a5bf0 con 0x55946862a710 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f56f7fff640 1 -- 172.21.15.132:0/3507433952 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5704046040 con 0x55946862a710 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f56f7fff640 1 -- 172.21.15.132:0/3507433952 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5710081020 con 0x55946862a710 2024-04-23T18:35:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f5721c16740 10 client.15270 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f5721c16740 10 client.15270 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f5721c16740 20 client.15270 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f5721c16740 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x5594686aa5b0 0x5594686a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f5721c16740 4 client.15270 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.485+0000 7f5721c16740 1 -- 172.21.15.132:0/3507433952 --> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] -- command(tid 0: {"prefix": "status"}) -- 0x5594685954e0 con 0x5594686aa5b0 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f571cca3640 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x5594686aa5b0 0x5594686a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f56f7fff640 10 client.15270 ms_handle_connect on v2:172.21.15.157:6832/1034092547 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f571cca3640 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x5594686aa5b0 0x5594686a92c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=40f242b9be2d7f56 server_cookie=d0984fad3ccff576 in_seq=0 out_seq=0 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f56f7fff640 1 -- 172.21.15.132:0/3507433952 <== mds.4 v2:172.21.15.157:6832/1034092547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5594685954e0 con 0x5594686aa5b0 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f56f7fff640 10 client.15270 handle_command_reply: tid=0 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f5721c16740 1 client.15270 shutdown 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f5721c16740 1 -- 172.21.15.132:0/3507433952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 msgr2=0x559468639560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f5721c16740 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x559468639560 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f571009bc40 tx=0x7f57100cd520 comp rx=0 tx=0).stop 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f5721c16740 1 -- 172.21.15.132:0/3507433952 shutdown_connections 2024-04-23T18:35:54.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f5721c16740 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x5594686aa5b0 0x5594686a92c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f5721c16740 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559468629d50 0x559468636540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f5721c16740 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55946862a710 0x559468639560 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f56f7fff640 0 client.15270 ms_handle_reset on v2:172.21.15.157:6832/1034092547 2024-04-23T18:35:54.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f5721c16740 1 --2- 172.21.15.132:0/3507433952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559468628b80 0x559468636000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.486+0000 7f5721c16740 1 -- 172.21.15.132:0/3507433952 >> 172.21.15.132:0/3507433952 conn(0x559468624070 msgr2=0x55946862d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f5721c16740 1 -- 172.21.15.132:0/3507433952 shutdown_connections 2024-04-23T18:35:54.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f5721c16740 1 -- 172.21.15.132:0/3507433952 wait complete. 2024-04-23T18:35:54.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f5721c16740 20 client.15270 trim_cache size 0 max 16384 2024-04-23T18:35:54.486 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:35:54.486 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:35:54.487 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 4, 2024-04-23T18:35:54.487 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24394, 2024-04-23T18:35:54.487 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:35:54.487 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:35:54.487 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:35:54.487 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 107.95306997, 2024-04-23T18:35:54.487 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 25, 2024-04-23T18:35:54.487 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 88, 2024-04-23T18:35:54.487 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:35:54.487 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 151.747846369 2024-04-23T18:35:54.487 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:35:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f571ef07640 1 -- 172.21.15.132:0/3465179751 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f57080a3c30 msgr2=0x7f57080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f571ef07640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f57080a3c30 0x7f57080a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5704006010 tx=0x7f57040095f0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f571ef07640 1 -- 172.21.15.132:0/3465179751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5718107cf0 msgr2=0x7f571819f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f571ef07640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5718107cf0 0x7f571819f750 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f571000b970 tx=0x7f571000be30 comp rx=0 tx=0).stop 2024-04-23T18:35:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f571ef07640 1 -- 172.21.15.132:0/3465179751 shutdown_connections 2024-04-23T18:35:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f571ef07640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f57080a3c30 0x7f57080a60f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f571ef07640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57181053c0 0x7f57181986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f571ef07640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5718107cf0 0x7f571819f750 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f571ef07640 1 --2- 172.21.15.132:0/3465179751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5718102a90 0x7f57181981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f571ef07640 1 -- 172.21.15.132:0/3465179751 >> 172.21.15.132:0/3465179751 conn(0x7f57180fc440 msgr2=0x7f57181033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f571ef07640 1 -- 172.21.15.132:0/3465179751 shutdown_connections 2024-04-23T18:35:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.487+0000 7f571ef07640 1 -- 172.21.15.132:0/3465179751 wait complete. 2024-04-23T18:35:54.490 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-23T18:35:54.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.599+0000 7f5309744640 1 -- 172.21.15.132:0/919826325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5304107d10 msgr2=0x7f530410a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.599+0000 7f5309744640 1 --2- 172.21.15.132:0/919826325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5304107d10 0x7f530410a100 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009ca0 tx=0x7f52fc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.599+0000 7f5309744640 1 -- 172.21.15.132:0/919826325 shutdown_connections 2024-04-23T18:35:54.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.599+0000 7f5309744640 1 --2- 172.21.15.132:0/919826325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5304107d10 0x7f530410a100 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.599+0000 7f5309744640 1 --2- 172.21.15.132:0/919826325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53041053e0 0x7f53041077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.599+0000 7f5309744640 1 --2- 172.21.15.132:0/919826325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5304102ab0 0x7f5304104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.599+0000 7f5309744640 1 -- 172.21.15.132:0/919826325 >> 172.21.15.132:0/919826325 conn(0x7f53040fc400 msgr2=0x7f53040fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.599+0000 7f5309744640 1 -- 172.21.15.132:0/919826325 shutdown_connections 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.599+0000 7f5309744640 1 -- 172.21.15.132:0/919826325 wait complete. 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5309744640 1 Processor -- start 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5309744640 1 -- start start 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5309744640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5304102ab0 0x7f5304198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5309744640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53041053e0 0x7f5304198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5309744640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5304107d10 0x7f530419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5309744640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f530410db50 con 0x7f5304102ab0 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5309744640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f530410d9d0 con 0x7f53041053e0 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5309744640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f530410dcd0 con 0x7f5304107d10 2024-04-23T18:35:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5302ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5304102ab0 0x7f5304198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5302ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5304102ab0 0x7f5304198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60182/0 (socket says 172.21.15.132:60182) 2024-04-23T18:35:54.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5302ffd640 1 -- 172.21.15.132:0/2797038046 learned_addr learned my addr 172.21.15.132:0/2797038046 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f52fbfff640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53041053e0 0x7f5304198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5302ffd640 1 -- 172.21.15.132:0/2797038046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5304107d10 msgr2=0x7f530419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5302ffd640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5304107d10 0x7f530419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5302ffd640 1 -- 172.21.15.132:0/2797038046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53041053e0 msgr2=0x7f5304198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5302ffd640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53041053e0 0x7f5304198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5302ffd640 1 -- 172.21.15.132:0/2797038046 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530419fe40 con 0x7f5304102ab0 2024-04-23T18:35:54.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.600+0000 7f5302ffd640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5304102ab0 0x7f5304198220 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f52f400d770 tx=0x7f52f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.601+0000 7f5300ff9640 1 -- 172.21.15.132:0/2797038046 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4017070 con 0x7f5304102ab0 2024-04-23T18:35:54.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.601+0000 7f5300ff9640 1 -- 172.21.15.132:0/2797038046 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f40054d0 con 0x7f5304102ab0 2024-04-23T18:35:54.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.601+0000 7f5300ff9640 1 -- 172.21.15.132:0/2797038046 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4004c30 con 0x7f5304102ab0 2024-04-23T18:35:54.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.601+0000 7f5309744640 1 -- 172.21.15.132:0/2797038046 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53041a0130 con 0x7f5304102ab0 2024-04-23T18:35:54.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.601+0000 7f5309744640 1 -- 172.21.15.132:0/2797038046 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53041a0700 con 0x7f5304102ab0 2024-04-23T18:35:54.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.603+0000 7f5300ff9640 1 -- 172.21.15.132:0/2797038046 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f52f4004e10 con 0x7f5304102ab0 2024-04-23T18:35:54.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.603+0000 7f530c468740 1 Processor -- start 2024-04-23T18:35:54.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.603+0000 7f5300ff9640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52f00a3c30 0x7f52f00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.603+0000 7f530c468740 1 -- start start 2024-04-23T18:35:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.603+0000 7f53037fe640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52f00a3c30 0x7f52f00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.603+0000 7f5300ff9640 1 -- 172.21.15.132:0/2797038046 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7f52f40cff20 con 0x7f5304102ab0 2024-04-23T18:35:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f530c468740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c0ffb80 0x563c9c0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f530c468740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c100d50 0x563c9c1011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f530c468740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c101710 0x563c9c107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f530c468740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x563c9c064c50 con 0x563c9c100d50 2024-04-23T18:35:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f530c468740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563c9c001830 con 0x563c9c101710 2024-04-23T18:35:54.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f530c468740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563c9c060c10 con 0x563c9c0ffb80 2024-04-23T18:35:54.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f52fbfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c100d50 0x563c9c1011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f52fbfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c100d50 0x563c9c1011d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60190/0 (socket says 172.21.15.132:60190) 2024-04-23T18:35:54.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f52fbfff640 1 -- 172.21.15.132:0/1908916281 learned_addr learned my addr 172.21.15.132:0/1908916281 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f5302ffd640 1 --2- 172.21.15.132:0/1908916281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c0ffb80 0x563c9c0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f53037fe640 1 --2- 172.21.15.132:0/1908916281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c101710 0x563c9c107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f53037fe640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52f00a3c30 0x7f52f00a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f52fc006010 tx=0x7f52fc0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f52fbfff640 1 -- 172.21.15.132:0/1908916281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c0ffb80 msgr2=0x563c9c0fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f52fbfff640 1 --2- 172.21.15.132:0/1908916281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c0ffb80 0x563c9c0fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f52fbfff640 1 -- 172.21.15.132:0/1908916281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c101710 msgr2=0x563c9c107fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f52fbfff640 1 --2- 172.21.15.132:0/1908916281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c101710 0x563c9c107fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f52fbfff640 1 -- 172.21.15.132:0/1908916281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c9bd772d0 con 0x563c9c100d50 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f52fbfff640 1 --2- 172.21.15.132:0/1908916281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c100d50 0x563c9c1011d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00b7c0 tx=0x7f52ec00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d657375299664e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f52f97fa640 1 -- 172.21.15.132:0/1908916281 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec00d950 con 0x563c9c100d50 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f52f97fa640 1 -- 172.21.15.132:0/1908916281 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ec018070 con 0x563c9c100d50 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.604+0000 7f52f97fa640 1 -- 172.21.15.132:0/1908916281 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0123e0 con 0x563c9c100d50 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.605+0000 7f530c468740 1 -- 172.21.15.132:0/1908916281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c100d50 msgr2=0x563c9c1011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.605+0000 7f530c468740 1 --2- 172.21.15.132:0/1908916281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c100d50 0x563c9c1011d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00b7c0 tx=0x7f52ec00bc80 comp rx=0 tx=0).stop 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.605+0000 7f530c468740 1 -- 172.21.15.132:0/1908916281 shutdown_connections 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.605+0000 7f530c468740 1 --2- 172.21.15.132:0/1908916281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c101710 0x563c9c107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.605+0000 7f530c468740 1 --2- 172.21.15.132:0/1908916281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c100d50 0x563c9c1011d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.605+0000 7f530c468740 1 --2- 172.21.15.132:0/1908916281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c0ffb80 0x563c9c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.605+0000 7f530c468740 1 -- 172.21.15.132:0/1908916281 >> 172.21.15.132:0/1908916281 conn(0x563c9c0fb070 msgr2=0x563c9c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.605+0000 7f530c468740 1 -- 172.21.15.132:0/1908916281 shutdown_connections 2024-04-23T18:35:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.605+0000 7f530c468740 1 -- 172.21.15.132:0/1908916281 wait complete. 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.605+0000 7f530c468740 1 -- start start 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.605+0000 7f530c468740 1 Processor -- start 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f530c468740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c0ffb80 0x563c9c10e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f530c468740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c100d50 0x563c9c10e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f530c468740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c101710 0x563c9c115970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f530c468740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x563c9c10cb30 con 0x563c9c101710 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f530c468740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563c9bfff470 con 0x563c9c0ffb80 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f530c468740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563c9c10ccb0 con 0x563c9c100d50 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f530c468740 10 client.0 _ll_register_callbacks cb 0x563c9c024f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f53037fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c101710 0x563c9c115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f53037fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c101710 0x563c9c115970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60202/0 (socket says 172.21.15.132:60202) 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f53037fe640 1 -- 172.21.15.132:0/2082790875 learned_addr learned my addr 172.21.15.132:0/2082790875 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f5302ffd640 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c0ffb80 0x563c9c10e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f52fbfff640 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c100d50 0x563c9c10e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f53037fe640 1 -- 172.21.15.132:0/2082790875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c100d50 msgr2=0x563c9c10e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f53037fe640 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c100d50 0x563c9c10e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f53037fe640 1 -- 172.21.15.132:0/2082790875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c0ffb80 msgr2=0x563c9c10e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f53037fe640 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c0ffb80 0x563c9c10e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f53037fe640 1 -- 172.21.15.132:0/2082790875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c9be416b0 con 0x563c9c101710 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f53037fe640 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c101710 0x563c9c115970 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f52fc031680 tx=0x7f52fc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d2a363f882c250 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f52ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f52ea7fc640 1 -- 172.21.15.132:0/2082790875 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc0045c0 con 0x563c9c101710 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f52ea7fc640 1 -- 172.21.15.132:0/2082790875 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7f52fc004760 con 0x563c9c101710 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f530c468740 1 -- 172.21.15.132:0/2082790875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52ec018070 con 0x563c9c101710 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f52ea7fc640 1 -- 172.21.15.132:0/2082790875 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f52fc058030 con 0x563c9c101710 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.606+0000 7f530c468740 10 client.15282 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.607+0000 7f530c468740 1 -- 172.21.15.132:0/2082790875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c9c118d50 con 0x563c9c101710 2024-04-23T18:35:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.607+0000 7f52ea7fc640 1 -- 172.21.15.132:0/2082790875 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ec018070 con 0x563c9c101710 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.607+0000 7f52ea7fc640 1 -- 172.21.15.132:0/2082790875 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f52fc030860 con 0x563c9c101710 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.607+0000 7f530c468740 10 client.15282 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.607+0000 7f530c468740 10 client.15282 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.607+0000 7f530c468740 20 client.15282 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.607+0000 7f530c468740 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x563c9c11b3f0 0x563c9c11c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.607+0000 7f530c468740 4 client.15282 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.607+0000 7f530c468740 1 -- 172.21.15.132:0/2082790875 --> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c9c072670 con 0x563c9c11b3f0 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.607+0000 7f5302ffd640 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x563c9c11b3f0 0x563c9c11c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.607+0000 7f5302ffd640 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x563c9c11b3f0 0x563c9c11c860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8b2a4b62def78d7 server_cookie=e62ea82ee99068d5 in_seq=0 out_seq=0 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.607+0000 7f52ea7fc640 10 client.15282 ms_handle_connect on v2:172.21.15.148:6832/3556073608 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f52ea7fc640 1 -- 172.21.15.132:0/2082790875 <== mds.3 v2:172.21.15.148:6832/3556073608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c9c072670 con 0x563c9c11b3f0 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f52ea7fc640 10 client.15282 handle_command_reply: tid=0 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f530c468740 1 client.15282 shutdown 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f530c468740 1 -- 172.21.15.132:0/2082790875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c101710 msgr2=0x563c9c115970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f530c468740 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c101710 0x563c9c115970 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f52fc031680 tx=0x7f52fc0095c0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f530c468740 1 -- 172.21.15.132:0/2082790875 shutdown_connections 2024-04-23T18:35:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f530c468740 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x563c9c11b3f0 0x563c9c11c860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f530c468740 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c100d50 0x563c9c10e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f530c468740 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c101710 0x563c9c115970 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f52ea7fc640 0 client.15282 ms_handle_reset on v2:172.21.15.148:6832/3556073608 2024-04-23T18:35:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f530c468740 1 --2- 172.21.15.132:0/2082790875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c0ffb80 0x563c9c10e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f530c468740 1 -- 172.21.15.132:0/2082790875 >> 172.21.15.132:0/2082790875 conn(0x563c9c0fb070 msgr2=0x563c9c0fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f530c468740 1 -- 172.21.15.132:0/2082790875 shutdown_connections 2024-04-23T18:35:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f530c468740 1 -- 172.21.15.132:0/2082790875 wait complete. 2024-04-23T18:35:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.608+0000 7f530c468740 20 client.15282 trim_cache size 0 max 16384 2024-04-23T18:35:54.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:54 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3637376288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:35:54.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:54 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1258418821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:54.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:54 smithi132 ceph-mon[29178]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:54.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:54 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-23T18:35:54.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:54 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/769181272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:54.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f530c468740 1 Processor -- start 2024-04-23T18:35:54.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f530c468740 1 -- start start 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f530c468740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 0x563c9c0f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f530c468740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c100d50 0x563c9c115930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f530c468740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c101710 0x563c9c1047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f530c468740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x563c9c11aba0 con 0x563c9c0ffb80 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f530c468740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563c9c11b270 con 0x563c9c100d50 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f530c468740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563c9c11b940 con 0x563c9c101710 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f5302ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 0x563c9c0f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f5302ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 0x563c9c0f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60214/0 (socket says 172.21.15.132:60214) 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f5302ffd640 1 -- 172.21.15.132:0/2410085133 learned_addr learned my addr 172.21.15.132:0/2410085133 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f52fbfff640 1 --2- 172.21.15.132:0/2410085133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c100d50 0x563c9c115930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f5302ffd640 1 -- 172.21.15.132:0/2410085133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c101710 msgr2=0x563c9c1047f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f5302ffd640 1 --2- 172.21.15.132:0/2410085133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c101710 0x563c9c1047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f5302ffd640 1 -- 172.21.15.132:0/2410085133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c100d50 msgr2=0x563c9c115930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f5302ffd640 1 --2- 172.21.15.132:0/2410085133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c100d50 0x563c9c115930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f5302ffd640 1 -- 172.21.15.132:0/2410085133 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c9c0fbd40 con 0x563c9c0ffb80 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.627+0000 7f5302ffd640 1 --2- 172.21.15.132:0/2410085133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 0x563c9c0f60b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009610 tx=0x7f52f401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5233209fecaef98 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.628+0000 7f52f8ff9640 1 -- 172.21.15.132:0/2410085133 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40b4070 con 0x563c9c0ffb80 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.628+0000 7f52f8ff9640 1 -- 172.21.15.132:0/2410085133 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f4009c30 con 0x563c9c0ffb80 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.628+0000 7f52f8ff9640 1 -- 172.21.15.132:0/2410085133 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4020040 con 0x563c9c0ffb80 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.628+0000 7f530c468740 1 -- 172.21.15.132:0/2410085133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 msgr2=0x563c9c0f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.628+0000 7f530c468740 1 --2- 172.21.15.132:0/2410085133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 0x563c9c0f60b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009610 tx=0x7f52f401d040 comp rx=0 tx=0).stop 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.628+0000 7f530c468740 1 -- 172.21.15.132:0/2410085133 shutdown_connections 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.628+0000 7f530c468740 1 --2- 172.21.15.132:0/2410085133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c101710 0x563c9c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.628+0000 7f530c468740 1 --2- 172.21.15.132:0/2410085133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c100d50 0x563c9c115930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.628+0000 7f530c468740 1 --2- 172.21.15.132:0/2410085133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 0x563c9c0f60b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.628+0000 7f530c468740 1 -- 172.21.15.132:0/2410085133 >> 172.21.15.132:0/2410085133 conn(0x563c9c0fb070 msgr2=0x563c9c0fb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.628+0000 7f530c468740 1 -- 172.21.15.132:0/2410085133 shutdown_connections 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.628+0000 7f530c468740 1 -- 172.21.15.132:0/2410085133 wait complete. 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f530c468740 1 -- start start 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f530c468740 1 Processor -- start 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f530c468740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 0x563c9c110da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f530c468740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c100d50 0x563c9c1112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f530c468740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c101710 0x563c9c114300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f530c468740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x563c9c11c6e0 con 0x563c9c0ffb80 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f530c468740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563c9c11c010 con 0x563c9c100d50 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f530c468740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563c9c11cdb0 con 0x563c9c101710 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f530c468740 10 client.0 _ll_register_callbacks cb 0x7f52fc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f5302ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 0x563c9c110da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f5302ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 0x563c9c110da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60228/0 (socket says 172.21.15.132:60228) 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f5302ffd640 1 -- 172.21.15.132:0/2466787100 learned_addr learned my addr 172.21.15.132:0/2466787100 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f52fbfff640 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c100d50 0x563c9c1112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f5302ffd640 1 -- 172.21.15.132:0/2466787100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c101710 msgr2=0x563c9c114300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f53037fe640 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c101710 0x563c9c114300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f5302ffd640 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c101710 0x563c9c114300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f5302ffd640 1 -- 172.21.15.132:0/2466787100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c100d50 msgr2=0x563c9c1112e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f5302ffd640 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c100d50 0x563c9c1112e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f5302ffd640 1 -- 172.21.15.132:0/2466787100 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c9c1148f0 con 0x563c9c0ffb80 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f52fbfff640 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c100d50 0x563c9c1112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f53037fe640 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c101710 0x563c9c114300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f5302ffd640 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 0x563c9c110da0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f52f40cdd60 tx=0x7f52f4009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3d9ec1680f5337 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.629+0000 7f52ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f52ea7fc640 1 -- 172.21.15.132:0/2466787100 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40b4070 con 0x563c9c0ffb80 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f52ea7fc640 1 -- 172.21.15.132:0/2466787100 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7f52f40a0320 con 0x563c9c0ffb80 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f530c468740 1 -- 172.21.15.132:0/2466787100 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f4009c30 con 0x563c9c0ffb80 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f52ea7fc640 1 -- 172.21.15.132:0/2466787100 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f52f409c1d0 con 0x563c9c0ffb80 2024-04-23T18:35:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f530c468740 10 client.15288 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f530c468740 1 -- 172.21.15.132:0/2466787100 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c9c115bf0 con 0x563c9c0ffb80 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f52ea7fc640 1 -- 172.21.15.132:0/2466787100 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f4009c30 con 0x563c9c0ffb80 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f52ea7fc640 1 -- 172.21.15.132:0/2466787100 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f52f409b090 con 0x563c9c0ffb80 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f530c468740 10 client.15288 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f530c468740 10 client.15288 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f530c468740 20 client.15288 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f530c468740 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x563c9c1857e0 0x563c9c1844f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f530c468740 4 client.15288 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f530c468740 1 -- 172.21.15.132:0/2466787100 --> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] -- command(tid 0: {"prefix": "status"}) -- 0x563c9c06c4e0 con 0x563c9c1857e0 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.630+0000 7f52fbfff640 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x563c9c1857e0 0x563c9c1844f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f52ea7fc640 10 client.15288 ms_handle_connect on v2:172.21.15.148:6832/3556073608 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f52fbfff640 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x563c9c1857e0 0x563c9c1844f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8dbcf9892c539988 server_cookie=1204a6321663f496 in_seq=0 out_seq=0 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f52ea7fc640 1 -- 172.21.15.132:0/2466787100 <== mds.3 v2:172.21.15.148:6832/3556073608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563c9c06c4e0 con 0x563c9c1857e0 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f52ea7fc640 10 client.15288 handle_command_reply: tid=0 2024-04-23T18:35:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f530c468740 1 client.15288 shutdown 2024-04-23T18:35:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f530c468740 1 -- 172.21.15.132:0/2466787100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 msgr2=0x563c9c110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f530c468740 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 0x563c9c110da0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f52f40cdd60 tx=0x7f52f4009740 comp rx=0 tx=0).stop 2024-04-23T18:35:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f530c468740 1 -- 172.21.15.132:0/2466787100 shutdown_connections 2024-04-23T18:35:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f530c468740 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x563c9c1857e0 0x563c9c1844f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f530c468740 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c9c101710 0x563c9c114300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f530c468740 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c9c100d50 0x563c9c1112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f52ea7fc640 0 client.15288 ms_handle_reset on v2:172.21.15.148:6832/3556073608 2024-04-23T18:35:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f530c468740 1 --2- 172.21.15.132:0/2466787100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c9c0ffb80 0x563c9c110da0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f530c468740 1 -- 172.21.15.132:0/2466787100 >> 172.21.15.132:0/2466787100 conn(0x563c9c0fb070 msgr2=0x563c9c0f60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f530c468740 1 -- 172.21.15.132:0/2466787100 shutdown_connections 2024-04-23T18:35:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f530c468740 1 -- 172.21.15.132:0/2466787100 wait complete. 2024-04-23T18:35:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.631+0000 7f530c468740 20 client.15288 trim_cache size 0 max 16384 2024-04-23T18:35:54.631 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:35:54.631 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:35:54.631 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 3, 2024-04-23T18:35:54.631 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24425, 2024-04-23T18:35:54.631 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:35:54.631 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:35:54.631 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:35:54.631 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 109.110113586, 2024-04-23T18:35:54.631 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 25, 2024-04-23T18:35:54.631 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 89, 2024-04-23T18:35:54.632 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:35:54.632 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 148.99679555399999 2024-04-23T18:35:54.632 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:35:54.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.632+0000 7f5309744640 1 -- 172.21.15.132:0/2797038046 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52f00a3c30 msgr2=0x7f52f00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.632+0000 7f5309744640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52f00a3c30 0x7f52f00a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f52fc006010 tx=0x7f52fc0093f0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.632+0000 7f5309744640 1 -- 172.21.15.132:0/2797038046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5304102ab0 msgr2=0x7f5304198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.632+0000 7f5309744640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5304102ab0 0x7f5304198220 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f52f400d770 tx=0x7f52f400dc30 comp rx=0 tx=0).stop 2024-04-23T18:35:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.632+0000 7f5309744640 1 -- 172.21.15.132:0/2797038046 shutdown_connections 2024-04-23T18:35:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.632+0000 7f5309744640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f52f00a3c30 0x7f52f00a60f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.632+0000 7f5309744640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5304107d10 0x7f530419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.632+0000 7f5309744640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53041053e0 0x7f5304198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.632+0000 7f5309744640 1 --2- 172.21.15.132:0/2797038046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5304102ab0 0x7f5304198220 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.632+0000 7f5309744640 1 -- 172.21.15.132:0/2797038046 >> 172.21.15.132:0/2797038046 conn(0x7f53040fc400 msgr2=0x7f5304103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.632+0000 7f5309744640 1 -- 172.21.15.132:0/2797038046 shutdown_connections 2024-04-23T18:35:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.632+0000 7f5309744640 1 -- 172.21.15.132:0/2797038046 wait complete. 2024-04-23T18:35:54.635 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T18:35:54.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.743+0000 7f49976c2640 1 -- 172.21.15.132:0/272338551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4990100f30 msgr2=0x7f4990101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.743+0000 7f49976c2640 1 --2- 172.21.15.132:0/272338551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4990100f30 0x7f4990101350 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f498c009900 tx=0x7f498c02a410 comp rx=0 tx=0).stop 2024-04-23T18:35:54.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.743+0000 7f49976c2640 1 -- 172.21.15.132:0/272338551 shutdown_connections 2024-04-23T18:35:54.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.743+0000 7f49976c2640 1 --2- 172.21.15.132:0/272338551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4990102a40 0x7f4990109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.743+0000 7f49976c2640 1 --2- 172.21.15.132:0/272338551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4990102080 0x7f4990102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.743+0000 7f49976c2640 1 --2- 172.21.15.132:0/272338551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4990100f30 0x7f4990101350 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.743+0000 7f49976c2640 1 -- 172.21.15.132:0/272338551 >> 172.21.15.132:0/272338551 conn(0x7f49900fc4a0 msgr2=0x7f49900fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.743+0000 7f49976c2640 1 -- 172.21.15.132:0/272338551 shutdown_connections 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.743+0000 7f49976c2640 1 -- 172.21.15.132:0/272338551 wait complete. 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.743+0000 7f49976c2640 1 Processor -- start 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.743+0000 7f49976c2640 1 -- start start 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f49976c2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4990100f30 0x7f4990198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f49976c2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4990102080 0x7f4990198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f49976c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4990102a40 0x7f499019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f49976c2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f499010dbf0 con 0x7f4990102080 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f49976c2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f499010da70 con 0x7f4990100f30 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f49976c2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f499010dd70 con 0x7f4990102a40 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f4994c5d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4990102080 0x7f4990198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f4994c5d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4990102080 0x7f4990198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60244/0 (socket says 172.21.15.132:60244) 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f4994c5d640 1 -- 172.21.15.132:0/4286396654 learned_addr learned my addr 172.21.15.132:0/4286396654 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f499545e640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4990100f30 0x7f4990198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f4995c5f640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4990102a40 0x7f499019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f4994c5d640 1 -- 172.21.15.132:0/4286396654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4990102a40 msgr2=0x7f499019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f4994c5d640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4990102a40 0x7f499019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f4994c5d640 1 -- 172.21.15.132:0/4286396654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4990100f30 msgr2=0x7f4990198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f4994c5d640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4990100f30 0x7f4990198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f4994c5d640 1 -- 172.21.15.132:0/4286396654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499019fe30 con 0x7f4990102080 2024-04-23T18:35:54.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f4995c5f640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4990102a40 0x7f499019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:54.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f4994c5d640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4990102080 0x7f4990198750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f498000c970 tx=0x7f498000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f49867fc640 1 -- 172.21.15.132:0/4286396654 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980007c10 con 0x7f4990102080 2024-04-23T18:35:54.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f49867fc640 1 -- 172.21.15.132:0/4286396654 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4980007db0 con 0x7f4990102080 2024-04-23T18:35:54.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f49976c2640 1 -- 172.21.15.132:0/4286396654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49901a0120 con 0x7f4990102080 2024-04-23T18:35:54.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f49867fc640 1 -- 172.21.15.132:0/4286396654 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498000f870 con 0x7f4990102080 2024-04-23T18:35:54.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.744+0000 7f49976c2640 1 -- 172.21.15.132:0/4286396654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49901a0670 con 0x7f4990102080 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.747+0000 7f49867fc640 1 -- 172.21.15.132:0/4286396654 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f498000fa10 con 0x7f4990102080 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.747+0000 7f499a3d2740 1 Processor -- start 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.747+0000 7f499a3d2740 1 -- start start 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f499a3d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 0x555fa4d46fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f499a3d2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d47d50 0x555fa4d481d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f499a3d2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 0x555fa4d4efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f499a3d2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x555fa4cabc50 con 0x555fa4d48710 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f499a3d2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555fa4c48830 con 0x555fa4d47d50 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f499a3d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555fa4ca7c10 con 0x555fa4d46b80 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f4994c5d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 0x555fa4d4efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f4994c5d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 0x555fa4d4efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60248/0 (socket says 172.21.15.132:60248) 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f4994c5d640 1 -- 172.21.15.132:0/3515736739 learned_addr learned my addr 172.21.15.132:0/3515736739 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f49867fc640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f497c0a3bb0 0x7f497c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f4995c5f640 1 --2- 172.21.15.132:0/3515736739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 0x555fa4d46fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f4995c5f640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f497c0a3bb0 0x7f497c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f499545e640 1 --2- 172.21.15.132:0/3515736739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d47d50 0x555fa4d481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f49867fc640 1 -- 172.21.15.132:0/4286396654 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7f49800d07e0 con 0x7f4990102080 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f4994c5d640 1 -- 172.21.15.132:0/3515736739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 msgr2=0x555fa4d46fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f4994c5d640 1 --2- 172.21.15.132:0/3515736739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 0x555fa4d46fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f4994c5d640 1 -- 172.21.15.132:0/3515736739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d47d50 msgr2=0x555fa4d481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f4994c5d640 1 --2- 172.21.15.132:0/3515736739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d47d50 0x555fa4d481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f4994c5d640 1 -- 172.21.15.132:0/3515736739 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa49be2d0 con 0x555fa4d48710 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f4994c5d640 1 --2- 172.21.15.132:0/3515736739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 0x555fa4d4efa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f498009cab0 tx=0x7f4980000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d141936d0e37da server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f4995c5f640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f497c0a3bb0 0x7f497c0a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x555fa4d47c50 tx=0x7f49880093d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f497b7fe640 1 -- 172.21.15.132:0/3515736739 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498009d220 con 0x555fa4d48710 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f497b7fe640 1 -- 172.21.15.132:0/3515736739 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f498009de50 con 0x555fa4d48710 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.748+0000 7f497b7fe640 1 -- 172.21.15.132:0/3515736739 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980018ae0 con 0x555fa4d48710 2024-04-23T18:35:54.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.749+0000 7f499a3d2740 1 -- 172.21.15.132:0/3515736739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 msgr2=0x555fa4d4efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.749+0000 7f499a3d2740 1 --2- 172.21.15.132:0/3515736739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 0x555fa4d4efa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f498009cab0 tx=0x7f4980000bc0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.749+0000 7f499a3d2740 1 -- 172.21.15.132:0/3515736739 shutdown_connections 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.749+0000 7f499a3d2740 1 --2- 172.21.15.132:0/3515736739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 0x555fa4d4efa0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.749+0000 7f499a3d2740 1 --2- 172.21.15.132:0/3515736739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d47d50 0x555fa4d481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.749+0000 7f499a3d2740 1 --2- 172.21.15.132:0/3515736739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 0x555fa4d46fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.749+0000 7f499a3d2740 1 -- 172.21.15.132:0/3515736739 >> 172.21.15.132:0/3515736739 conn(0x555fa4d42070 msgr2=0x555fa4d42470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.749+0000 7f499a3d2740 1 -- 172.21.15.132:0/3515736739 shutdown_connections 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.749+0000 7f499a3d2740 1 -- 172.21.15.132:0/3515736739 wait complete. 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.749+0000 7f499a3d2740 1 -- start start 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f499a3d2740 1 Processor -- start 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f499a3d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 0x555fa4d51000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f499a3d2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d47d50 0x555fa4d51540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f499a3d2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 0x555fa4d58510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f499a3d2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x555fa4d4f6c0 con 0x555fa4d48710 2024-04-23T18:35:54.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f499a3d2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555fa4c46470 con 0x555fa4d47d50 2024-04-23T18:35:54.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f499a3d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555fa4d4f840 con 0x555fa4d46b80 2024-04-23T18:35:54.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f499a3d2740 10 client.0 _ll_register_callbacks cb 0x555fa4c6bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:54.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f4995c5f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 0x555fa4d58510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f4995c5f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 0x555fa4d58510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60250/0 (socket says 172.21.15.132:60250) 2024-04-23T18:35:54.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f4995c5f640 1 -- 172.21.15.132:0/720105573 learned_addr learned my addr 172.21.15.132:0/720105573 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f499545e640 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 0x555fa4d51000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f4994c5d640 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d47d50 0x555fa4d51540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f4995c5f640 1 -- 172.21.15.132:0/720105573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 msgr2=0x555fa4d51000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f4995c5f640 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 0x555fa4d51000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f4995c5f640 1 -- 172.21.15.132:0/720105573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d47d50 msgr2=0x555fa4d51540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f4995c5f640 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d47d50 0x555fa4d51540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f4995c5f640 1 -- 172.21.15.132:0/720105573 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa4a886b0 con 0x555fa4d48710 2024-04-23T18:35:54.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f4995c5f640 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 0x555fa4d58510 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4988016ba0 tx=0x7f4988004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c914542386f1db82 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.750+0000 7f4978ff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:54.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f4978ff9640 1 -- 172.21.15.132:0/720105573 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4988029060 con 0x555fa4d48710 2024-04-23T18:35:54.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f4978ff9640 1 -- 172.21.15.132:0/720105573 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7f498801b9d0 con 0x555fa4d48710 2024-04-23T18:35:54.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f499a3d2740 1 -- 172.21.15.132:0/720105573 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498009de50 con 0x555fa4d48710 2024-04-23T18:35:54.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f4978ff9640 1 -- 172.21.15.132:0/720105573 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4988013040 con 0x555fa4d48710 2024-04-23T18:35:54.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f499a3d2740 10 client.15300 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:54.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f499a3d2740 1 -- 172.21.15.132:0/720105573 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555fa4d5b8f0 con 0x555fa4d48710 2024-04-23T18:35:54.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f4978ff9640 1 -- 172.21.15.132:0/720105573 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f498009de50 con 0x555fa4d48710 2024-04-23T18:35:54.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f4978ff9640 1 -- 172.21.15.132:0/720105573 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f498802fb60 con 0x555fa4d48710 2024-04-23T18:35:54.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f499a3d2740 10 client.15300 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:54.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f499a3d2740 10 client.15300 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:35:54.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f499a3d2740 20 client.15300 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:54.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f499a3d2740 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x555fa4d5dfa0 0x555fa4d5f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f499a3d2740 4 client.15300 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:54.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f499a3d2740 1 -- 172.21.15.132:0/720105573 --> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fa4cb9670 con 0x555fa4d5dfa0 2024-04-23T18:35:54.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.751+0000 7f499545e640 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x555fa4d5dfa0 0x555fa4d5f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.752+0000 7f4978ff9640 10 client.15300 ms_handle_connect on v2:172.21.15.157:6838/1528717147 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.752+0000 7f499545e640 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x555fa4d5dfa0 0x555fa4d5f410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0124703bfa2134e server_cookie=d2587d00958ecbe5 in_seq=0 out_seq=0 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.752+0000 7f4978ff9640 1 -- 172.21.15.132:0/720105573 <== mds.2 v2:172.21.15.157:6838/1528717147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fa4cb9670 con 0x555fa4d5dfa0 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.752+0000 7f4978ff9640 10 client.15300 handle_command_reply: tid=0 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f499a3d2740 1 client.15300 shutdown 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f499a3d2740 1 -- 172.21.15.132:0/720105573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 msgr2=0x555fa4d58510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f499a3d2740 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 0x555fa4d58510 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4988016ba0 tx=0x7f4988004b50 comp rx=0 tx=0).stop 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f499a3d2740 1 -- 172.21.15.132:0/720105573 shutdown_connections 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f499a3d2740 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x555fa4d5dfa0 0x555fa4d5f410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f499a3d2740 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d48710 0x555fa4d58510 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f4978ff9640 0 client.15300 ms_handle_reset on v2:172.21.15.157:6838/1528717147 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f499a3d2740 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d47d50 0x555fa4d51540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f499a3d2740 1 --2- 172.21.15.132:0/720105573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 0x555fa4d51000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f499a3d2740 1 -- 172.21.15.132:0/720105573 >> 172.21.15.132:0/720105573 conn(0x555fa4d42070 msgr2=0x555fa4d411d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f499a3d2740 1 -- 172.21.15.132:0/720105573 shutdown_connections 2024-04-23T18:35:54.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f499a3d2740 1 -- 172.21.15.132:0/720105573 wait complete. 2024-04-23T18:35:54.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.753+0000 7f499a3d2740 20 client.15300 trim_cache size 0 max 16384 2024-04-23T18:35:54.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f499a3d2740 1 Processor -- start 2024-04-23T18:35:54.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f499a3d2740 1 -- start start 2024-04-23T18:35:54.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f499a3d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 0x555fa4d3d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f499a3d2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 0x555fa4d51000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f499a3d2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d48710 0x555fa4d51540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f499a3d2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x555fa4d5d750 con 0x555fa4d47d50 2024-04-23T18:35:54.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f499a3d2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555fa4d5de20 con 0x555fa4d48710 2024-04-23T18:35:54.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f499a3d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555fa4d5e4f0 con 0x555fa4d46b80 2024-04-23T18:35:54.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f4994c5d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 0x555fa4d51000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f4994c5d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 0x555fa4d51000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60254/0 (socket says 172.21.15.132:60254) 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f4994c5d640 1 -- 172.21.15.132:0/1732459491 learned_addr learned my addr 172.21.15.132:0/1732459491 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f4995c5f640 1 --2- 172.21.15.132:0/1732459491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d48710 0x555fa4d51540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f499545e640 1 --2- 172.21.15.132:0/1732459491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 0x555fa4d3d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f4994c5d640 1 -- 172.21.15.132:0/1732459491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 msgr2=0x555fa4d3d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f4994c5d640 1 --2- 172.21.15.132:0/1732459491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 0x555fa4d3d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f4994c5d640 1 -- 172.21.15.132:0/1732459491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d48710 msgr2=0x555fa4d51540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f4994c5d640 1 --2- 172.21.15.132:0/1732459491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d48710 0x555fa4d51540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f4994c5d640 1 -- 172.21.15.132:0/1732459491 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa4d477d0 con 0x555fa4d47d50 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.771+0000 7f4995c5f640 1 --2- 172.21.15.132:0/1732459491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d48710 0x555fa4d51540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f4994c5d640 1 --2- 172.21.15.132:0/1732459491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 0x555fa4d51000 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f498009c820 tx=0x7f49800a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cd67fb1d962bae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f497affd640 1 -- 172.21.15.132:0/1732459491 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498009d220 con 0x555fa4d47d50 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f497affd640 1 -- 172.21.15.132:0/1732459491 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f498009de50 con 0x555fa4d47d50 2024-04-23T18:35:54.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f497affd640 1 -- 172.21.15.132:0/1732459491 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498000fd20 con 0x555fa4d47d50 2024-04-23T18:35:54.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f499a3d2740 1 -- 172.21.15.132:0/1732459491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 msgr2=0x555fa4d51000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f499a3d2740 1 --2- 172.21.15.132:0/1732459491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 0x555fa4d51000 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f498009c820 tx=0x7f49800a5700 comp rx=0 tx=0).stop 2024-04-23T18:35:54.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f499a3d2740 1 -- 172.21.15.132:0/1732459491 shutdown_connections 2024-04-23T18:35:54.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f499a3d2740 1 --2- 172.21.15.132:0/1732459491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d48710 0x555fa4d51540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f499a3d2740 1 --2- 172.21.15.132:0/1732459491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 0x555fa4d51000 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f499a3d2740 1 --2- 172.21.15.132:0/1732459491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d46b80 0x555fa4d3d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f499a3d2740 1 -- 172.21.15.132:0/1732459491 >> 172.21.15.132:0/1732459491 conn(0x555fa4d42070 msgr2=0x555fa4d42450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f499a3d2740 1 -- 172.21.15.132:0/1732459491 shutdown_connections 2024-04-23T18:35:54.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.772+0000 7f499a3d2740 1 -- 172.21.15.132:0/1732459491 wait complete. 2024-04-23T18:35:54.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f499a3d2740 1 -- start start 2024-04-23T18:35:54.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f499a3d2740 1 Processor -- start 2024-04-23T18:35:54.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f499a3d2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d46b80 0x555fa4d57bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f499a3d2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 0x555fa4d58130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f499a3d2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d48710 0x555fa4dc64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f499a3d2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x555fa4d5f290 con 0x555fa4d47d50 2024-04-23T18:35:54.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f499a3d2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555fa4d5ebc0 con 0x555fa4d46b80 2024-04-23T18:35:54.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f499a3d2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555fa4d5f960 con 0x555fa4d48710 2024-04-23T18:35:54.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f499a3d2740 10 client.0 _ll_register_callbacks cb 0x7f4988009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:54.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f4994c5d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 0x555fa4d58130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f4994c5d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 0x555fa4d58130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60268/0 (socket says 172.21.15.132:60268) 2024-04-23T18:35:54.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f4994c5d640 1 -- 172.21.15.132:0/3430950901 learned_addr learned my addr 172.21.15.132:0/3430950901 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f499545e640 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d46b80 0x555fa4d57bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f4994c5d640 1 -- 172.21.15.132:0/3430950901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d48710 msgr2=0x555fa4dc64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f4994c5d640 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d48710 0x555fa4dc64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f4994c5d640 1 -- 172.21.15.132:0/3430950901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d46b80 msgr2=0x555fa4d57bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f4994c5d640 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d46b80 0x555fa4d57bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f4994c5d640 1 -- 172.21.15.132:0/3430950901 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa4dc6aa0 con 0x555fa4d47d50 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f499545e640 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d46b80 0x555fa4d57bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f4994c5d640 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 0x555fa4d58130 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f49800994f0 tx=0x7f498009ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550c2df9510d5475 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.773+0000 7f4978ff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f4978ff9640 1 -- 172.21.15.132:0/3430950901 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980065060 con 0x555fa4d47d50 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f4978ff9640 1 -- 172.21.15.132:0/3430950901 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7f49800a5270 con 0x555fa4d47d50 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f499a3d2740 1 -- 172.21.15.132:0/3430950901 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498009de50 con 0x555fa4d47d50 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f4978ff9640 1 -- 172.21.15.132:0/3430950901 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f49800bbc50 con 0x555fa4d47d50 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f499a3d2740 10 client.15306 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f499a3d2740 1 -- 172.21.15.132:0/3430950901 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555fa4dc7da0 con 0x555fa4d47d50 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f4978ff9640 1 -- 172.21.15.132:0/3430950901 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f498009de50 con 0x555fa4d47d50 2024-04-23T18:35:54.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f4978ff9640 1 -- 172.21.15.132:0/3430950901 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4980061af0 con 0x555fa4d47d50 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f499a3d2740 10 client.15306 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f499a3d2740 10 client.15306 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f499a3d2740 20 client.15306 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f499a3d2740 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x555fa4dcc290 0x555fa4dcafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f499a3d2740 4 client.15306 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f499a3d2740 1 -- 172.21.15.132:0/3430950901 --> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] -- command(tid 0: {"prefix": "status"}) -- 0x555fa4cb34e0 con 0x555fa4dcc290 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.774+0000 7f499545e640 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x555fa4dcc290 0x555fa4dcafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f4978ff9640 10 client.15306 ms_handle_connect on v2:172.21.15.157:6838/1528717147 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499545e640 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x555fa4dcc290 0x555fa4dcafa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=629e1c83ea4d35e9 server_cookie=ee05ffc9dfa4566 in_seq=0 out_seq=0 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f4978ff9640 1 -- 172.21.15.132:0/3430950901 <== mds.2 v2:172.21.15.157:6838/1528717147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555fa4cb34e0 con 0x555fa4dcc290 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f4978ff9640 10 client.15306 handle_command_reply: tid=0 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499a3d2740 1 client.15306 shutdown 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499a3d2740 1 -- 172.21.15.132:0/3430950901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 msgr2=0x555fa4d58130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499a3d2740 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 0x555fa4d58130 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f49800994f0 tx=0x7f498009ae90 comp rx=0 tx=0).stop 2024-04-23T18:35:54.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499a3d2740 1 -- 172.21.15.132:0/3430950901 shutdown_connections 2024-04-23T18:35:54.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499a3d2740 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x555fa4dcc290 0x555fa4dcafa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499a3d2740 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa4d48710 0x555fa4dc64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499a3d2740 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fa4d47d50 0x555fa4d58130 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f4978ff9640 0 client.15306 ms_handle_reset on v2:172.21.15.157:6838/1528717147 2024-04-23T18:35:54.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499a3d2740 1 --2- 172.21.15.132:0/3430950901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555fa4d46b80 0x555fa4d57bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499a3d2740 1 -- 172.21.15.132:0/3430950901 >> 172.21.15.132:0/3430950901 conn(0x555fa4d42070 msgr2=0x555fa4d3d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499a3d2740 1 -- 172.21.15.132:0/3430950901 shutdown_connections 2024-04-23T18:35:54.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499a3d2740 1 -- 172.21.15.132:0/3430950901 wait complete. 2024-04-23T18:35:54.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.775+0000 7f499a3d2740 20 client.15306 trim_cache size 0 max 16384 2024-04-23T18:35:54.775 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:35:54.776 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:35:54.776 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 2, 2024-04-23T18:35:54.776 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24460, 2024-04-23T18:35:54.776 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:35:54.776 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:35:54.776 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:35:54.776 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 110.235522921, 2024-04-23T18:35:54.776 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 25, 2024-04-23T18:35:54.776 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 89, 2024-04-23T18:35:54.776 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:35:54.776 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 128.93570985400001 2024-04-23T18:35:54.776 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:35:54.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.776+0000 7f49976c2640 1 -- 172.21.15.132:0/4286396654 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f497c0a3bb0 msgr2=0x7f497c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.776+0000 7f49976c2640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f497c0a3bb0 0x7f497c0a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x555fa4d47c50 tx=0x7f49880093d0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.776+0000 7f49976c2640 1 -- 172.21.15.132:0/4286396654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4990102080 msgr2=0x7f4990198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.776+0000 7f49976c2640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4990102080 0x7f4990198750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f498000c970 tx=0x7f498000ce30 comp rx=0 tx=0).stop 2024-04-23T18:35:54.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.776+0000 7f49976c2640 1 -- 172.21.15.132:0/4286396654 shutdown_connections 2024-04-23T18:35:54.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.776+0000 7f49976c2640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f497c0a3bb0 0x7f497c0a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.776+0000 7f49976c2640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4990102a40 0x7f499019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.776+0000 7f49976c2640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4990102080 0x7f4990198750 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.776+0000 7f49976c2640 1 --2- 172.21.15.132:0/4286396654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4990100f30 0x7f4990198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.776+0000 7f49976c2640 1 -- 172.21.15.132:0/4286396654 >> 172.21.15.132:0/4286396654 conn(0x7f49900fc4a0 msgr2=0x7f49901053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.776+0000 7f49976c2640 1 -- 172.21.15.132:0/4286396654 shutdown_connections 2024-04-23T18:35:54.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.776+0000 7f49976c2640 1 -- 172.21.15.132:0/4286396654 wait complete. 2024-04-23T18:35:54.779 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T18:35:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 -- 172.21.15.132:0/1733390431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc728101fe0 msgr2=0x7fc728102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 --2- 172.21.15.132:0/1733390431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc728101fe0 0x7fc728102460 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc718009900 tx=0x7fc71802a410 comp rx=0 tx=0).stop 2024-04-23T18:35:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 -- 172.21.15.132:0/1733390431 shutdown_connections 2024-04-23T18:35:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 --2- 172.21.15.132:0/1733390431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7281029a0 0x7fc7281092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 --2- 172.21.15.132:0/1733390431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc728101fe0 0x7fc728102460 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 --2- 172.21.15.132:0/1733390431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc728100e90 0x7fc7281012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 -- 172.21.15.132:0/1733390431 >> 172.21.15.132:0/1733390431 conn(0x7fc7280fc400 msgr2=0x7fc7280fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 -- 172.21.15.132:0/1733390431 shutdown_connections 2024-04-23T18:35:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 -- 172.21.15.132:0/1733390431 wait complete. 2024-04-23T18:35:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 Processor -- start 2024-04-23T18:35:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 -- start start 2024-04-23T18:35:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc728100e90 0x7fc72806b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc728101fe0 0x7fc72819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7281029a0 0x7fc72819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc72810c9e0 con 0x7fc728100e90 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc72810c860 con 0x7fc7281029a0 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72f420640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc72810cb60 con 0x7fc728101fe0 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.887+0000 7fc72d1bc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc728100e90 0x7fc72806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc72d1bc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc728100e90 0x7fc72806b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60296/0 (socket says 172.21.15.132:60296) 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc72d1bc640 1 -- 172.21.15.132:0/2327777591 learned_addr learned my addr 172.21.15.132:0/2327777591 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc72d1bc640 1 -- 172.21.15.132:0/2327777591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc728101fe0 msgr2=0x7fc72819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/2327777591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc728101fe0 0x7fc72819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc72d1bc640 1 -- 172.21.15.132:0/2327777591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7281029a0 msgr2=0x7fc72819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/2327777591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7281029a0 0x7fc72819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc72d1bc640 1 -- 172.21.15.132:0/2327777591 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72819fe80 con 0x7fc728100e90 2024-04-23T18:35:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/2327777591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc728100e90 0x7fc72806b120 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc72400d950 tx=0x7fc72400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc71e7fc640 1 -- 172.21.15.132:0/2327777591 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724017070 con 0x7fc728100e90 2024-04-23T18:35:54.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc71e7fc640 1 -- 172.21.15.132:0/2327777591 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc724009c30 con 0x7fc728100e90 2024-04-23T18:35:54.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc71e7fc640 1 -- 172.21.15.132:0/2327777591 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72400f620 con 0x7fc728100e90 2024-04-23T18:35:54.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc72f420640 1 -- 172.21.15.132:0/2327777591 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7281a0110 con 0x7fc728100e90 2024-04-23T18:35:54.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.888+0000 7fc72f420640 1 -- 172.21.15.132:0/2327777591 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7281a0660 con 0x7fc728100e90 2024-04-23T18:35:54.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.890+0000 7fc71e7fc640 1 -- 172.21.15.132:0/2327777591 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fc724010460 con 0x7fc728100e90 2024-04-23T18:35:54.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc73220b740 1 Processor -- start 2024-04-23T18:35:54.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc73220b740 1 -- start start 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc73220b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525ab80 0x55b01525afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc73220b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525bd50 0x55b01525c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc73220b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525c710 0x55b015262fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc73220b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b0151bfc50 con 0x55b01525c710 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc73220b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b01515c830 con 0x55b01525bd50 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc73220b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b0151bbc10 con 0x55b01525ab80 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc72c9bb640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525c710 0x55b015262fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc72c9bb640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525c710 0x55b015262fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60302/0 (socket says 172.21.15.132:60302) 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc72c9bb640 1 -- 172.21.15.132:0/1045350081 learned_addr learned my addr 172.21.15.132:0/1045350081 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc71e7fc640 1 --2- 172.21.15.132:0/2327777591 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc7140a3bb0 0x7fc7140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/1045350081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525bd50 0x55b01525c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc72d9bd640 1 --2- 172.21.15.132:0/2327777591 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc7140a3bb0 0x7fc7140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc72c9bb640 1 -- 172.21.15.132:0/1045350081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525ab80 msgr2=0x55b01525afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc72c9bb640 1 --2- 172.21.15.132:0/1045350081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525ab80 0x55b01525afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc72c9bb640 1 -- 172.21.15.132:0/1045350081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525bd50 msgr2=0x55b01525c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc72c9bb640 1 --2- 172.21.15.132:0/1045350081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525bd50 0x55b01525c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc72c9bb640 1 -- 172.21.15.132:0/1045350081 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b014ed22d0 con 0x55b01525c710 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.891+0000 7fc71e7fc640 1 -- 172.21.15.132:0/2327777591 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7fc7240d0110 con 0x7fc728100e90 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.892+0000 7fc72c9bb640 1 --2- 172.21.15.132:0/1045350081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525c710 0x55b015262fa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc71802a930 tx=0x7fc718005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aeaeb049e2cf2b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.892+0000 7fc72d9bd640 1 --2- 172.21.15.132:0/2327777591 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc7140a3bb0 0x7fc7140a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55b01525bc50 tx=0x7fc720009780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.892+0000 7fc7137fe640 1 -- 172.21.15.132:0/1045350081 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71803e070 con 0x55b01525c710 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.892+0000 7fc7137fe640 1 -- 172.21.15.132:0/1045350081 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc718004030 con 0x55b01525c710 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.892+0000 7fc7137fe640 1 -- 172.21.15.132:0/1045350081 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718035040 con 0x55b01525c710 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.892+0000 7fc73220b740 1 -- 172.21.15.132:0/1045350081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525c710 msgr2=0x55b015262fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.892+0000 7fc73220b740 1 --2- 172.21.15.132:0/1045350081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525c710 0x55b015262fa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc71802a930 tx=0x7fc718005e60 comp rx=0 tx=0).stop 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.892+0000 7fc73220b740 1 -- 172.21.15.132:0/1045350081 shutdown_connections 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.892+0000 7fc73220b740 1 --2- 172.21.15.132:0/1045350081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525c710 0x55b015262fa0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.892+0000 7fc73220b740 1 --2- 172.21.15.132:0/1045350081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525bd50 0x55b01525c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.892+0000 7fc73220b740 1 --2- 172.21.15.132:0/1045350081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525ab80 0x55b01525afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.892+0000 7fc73220b740 1 -- 172.21.15.132:0/1045350081 >> 172.21.15.132:0/1045350081 conn(0x55b015256070 msgr2=0x55b015256470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc73220b740 1 -- 172.21.15.132:0/1045350081 shutdown_connections 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc73220b740 1 -- 172.21.15.132:0/1045350081 wait complete. 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc73220b740 1 -- start start 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc73220b740 1 Processor -- start 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc73220b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525ab80 0x55b015264f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc73220b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525bd50 0x55b0152654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc73220b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525c710 0x55b01526c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc73220b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b0152636c0 con 0x55b01525bd50 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc73220b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b01515a470 con 0x55b01525ab80 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc73220b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b015263840 con 0x55b01525c710 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc73220b740 10 client.0 _ll_register_callbacks cb 0x55b01517ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:54.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc72c9bb640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525bd50 0x55b0152654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc72c9bb640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525bd50 0x55b0152654c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60316/0 (socket says 172.21.15.132:60316) 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc72c9bb640 1 -- 172.21.15.132:0/2158229149 learned_addr learned my addr 172.21.15.132:0/2158229149 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.893+0000 7fc72d9bd640 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525c710 0x55b01526c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525ab80 0x55b015264f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc72c9bb640 1 -- 172.21.15.132:0/2158229149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525c710 msgr2=0x55b01526c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc72c9bb640 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525c710 0x55b01526c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc72c9bb640 1 -- 172.21.15.132:0/2158229149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525ab80 msgr2=0x55b015264f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc72c9bb640 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525ab80 0x55b015264f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc72c9bb640 1 -- 172.21.15.132:0/2158229149 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b014f9c6b0 con 0x55b01525bd50 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc72d9bd640 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525c710 0x55b01526c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc72c9bb640 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525bd50 0x55b0152654c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc718005ab0 tx=0x7fc718005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313eb0a25082ec49 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc710ff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc710ff9640 1 -- 172.21.15.132:0/2158229149 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71804a030 con 0x55b01525bd50 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc710ff9640 1 -- 172.21.15.132:0/2158229149 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7fc7180041e0 con 0x55b01525bd50 2024-04-23T18:35:54.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc73220b740 1 -- 172.21.15.132:0/2158229149 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc718004030 con 0x55b01525bd50 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc710ff9640 1 -- 172.21.15.132:0/2158229149 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc718035070 con 0x55b01525bd50 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc73220b740 10 client.15318 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc73220b740 1 -- 172.21.15.132:0/2158229149 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b01526f870 con 0x55b01525bd50 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc710ff9640 1 -- 172.21.15.132:0/2158229149 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc718004030 con 0x55b01525bd50 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc710ff9640 1 -- 172.21.15.132:0/2158229149 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc718058070 con 0x55b01525bd50 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc73220b740 10 client.15318 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc73220b740 10 client.15318 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc73220b740 20 client.15318 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc73220b740 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55b015271a30 0x55b015272ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc73220b740 4 client.15318 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.894+0000 7fc73220b740 1 -- 172.21.15.132:0/2158229149 --> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0151cd670 con 0x55b015271a30 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.895+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55b015271a30 0x55b015272ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.895+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55b015271a30 0x55b015272ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74bda3a64991419f server_cookie=efe92fe82cef5949 in_seq=0 out_seq=0 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.895+0000 7fc710ff9640 10 client.15318 ms_handle_connect on v2:172.21.15.148:6836/2626530887 2024-04-23T18:35:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc710ff9640 1 -- 172.21.15.132:0/2158229149 <== mds.1 v2:172.21.15.148:6836/2626530887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fc724096ae0 con 0x55b015271a30 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc710ff9640 10 client.15318 handle_command_reply: tid=0 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc73220b740 1 client.15318 shutdown 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc73220b740 1 -- 172.21.15.132:0/2158229149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525bd50 msgr2=0x55b0152654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc73220b740 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525bd50 0x55b0152654c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc718005ab0 tx=0x7fc718005630 comp rx=0 tx=0).stop 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc73220b740 1 -- 172.21.15.132:0/2158229149 shutdown_connections 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc73220b740 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55b015271a30 0x55b015272ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc73220b740 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525c710 0x55b01526c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc710ff9640 0 client.15318 ms_handle_reset on v2:172.21.15.148:6836/2626530887 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc73220b740 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525bd50 0x55b0152654c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc73220b740 1 --2- 172.21.15.132:0/2158229149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525ab80 0x55b015264f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc73220b740 1 -- 172.21.15.132:0/2158229149 >> 172.21.15.132:0/2158229149 conn(0x55b015256070 msgr2=0x55b0152551d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc73220b740 1 -- 172.21.15.132:0/2158229149 shutdown_connections 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc73220b740 1 -- 172.21.15.132:0/2158229149 wait complete. 2024-04-23T18:35:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.896+0000 7fc73220b740 20 client.15318 trim_cache size 0 max 16384 2024-04-23T18:35:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc73220b740 1 Processor -- start 2024-04-23T18:35:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc73220b740 1 -- start start 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc73220b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 0x55b01526cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc73220b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525bd50 0x55b015263be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc73220b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525c710 0x55b0152510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc73220b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b0152711e0 con 0x55b01525ab80 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc73220b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b0152718b0 con 0x55b01525c710 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc73220b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b015271f80 con 0x55b01525bd50 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc72d1bc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 0x55b01526cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc72d1bc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 0x55b01526cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60322/0 (socket says 172.21.15.132:60322) 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc72d1bc640 1 -- 172.21.15.132:0/2028951481 learned_addr learned my addr 172.21.15.132:0/2028951481 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc72c9bb640 1 --2- 172.21.15.132:0/2028951481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525bd50 0x55b015263be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc72d1bc640 1 -- 172.21.15.132:0/2028951481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525bd50 msgr2=0x55b015263be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/2028951481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525bd50 0x55b015263be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc72d9bd640 1 --2- 172.21.15.132:0/2028951481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525c710 0x55b0152510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc72d1bc640 1 -- 172.21.15.132:0/2028951481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525c710 msgr2=0x55b0152510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/2028951481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525c710 0x55b0152510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc72d1bc640 1 -- 172.21.15.132:0/2028951481 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b015256d40 con 0x55b01525ab80 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.914+0000 7fc72c9bb640 1 --2- 172.21.15.132:0/2028951481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525bd50 0x55b015263be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/2028951481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 0x55b01526cf50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc7240cdbe0 tx=0x7fc72406de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0b823f48fb8ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc712ffd640 1 -- 172.21.15.132:0/2028951481 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724020040 con 0x55b01525ab80 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc712ffd640 1 -- 172.21.15.132:0/2028951481 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b0151cd670 con 0x55b01525ab80 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc712ffd640 1 -- 172.21.15.132:0/2028951481 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72409a6c0 con 0x55b01525ab80 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc73220b740 1 -- 172.21.15.132:0/2028951481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 msgr2=0x55b01526cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc73220b740 1 --2- 172.21.15.132:0/2028951481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 0x55b01526cf50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc7240cdbe0 tx=0x7fc72406de50 comp rx=0 tx=0).stop 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc73220b740 1 -- 172.21.15.132:0/2028951481 shutdown_connections 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc73220b740 1 --2- 172.21.15.132:0/2028951481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525bd50 0x55b015263be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc73220b740 1 --2- 172.21.15.132:0/2028951481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525c710 0x55b0152510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc73220b740 1 --2- 172.21.15.132:0/2028951481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 0x55b01526cf50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc73220b740 1 -- 172.21.15.132:0/2028951481 >> 172.21.15.132:0/2028951481 conn(0x55b015256070 msgr2=0x55b015256450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc73220b740 1 -- 172.21.15.132:0/2028951481 shutdown_connections 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.915+0000 7fc73220b740 1 -- 172.21.15.132:0/2028951481 wait complete. 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc73220b740 1 -- start start 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc73220b740 1 Processor -- start 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc73220b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 0x55b0152679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc73220b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525bd50 0x55b015267f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc73220b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525c710 0x55b01526af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc73220b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b015272d20 con 0x55b01525ab80 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc73220b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b015272650 con 0x55b01525bd50 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc73220b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b0152733f0 con 0x55b01525c710 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc73220b740 10 client.0 _ll_register_callbacks cb 0x7fc718009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc72d1bc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 0x55b0152679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc72d1bc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 0x55b0152679d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60338/0 (socket says 172.21.15.132:60338) 2024-04-23T18:35:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc72d1bc640 1 -- 172.21.15.132:0/397316702 learned_addr learned my addr 172.21.15.132:0/397316702 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc72d1bc640 1 -- 172.21.15.132:0/397316702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525c710 msgr2=0x55b01526af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/397316702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525c710 0x55b01526af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc72d1bc640 1 -- 172.21.15.132:0/397316702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525bd50 msgr2=0x55b015267f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc72c9bb640 1 --2- 172.21.15.132:0/397316702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525bd50 0x55b015267f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/397316702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525bd50 0x55b015267f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc72d1bc640 1 -- 172.21.15.132:0/397316702 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b01526b520 con 0x55b01525ab80 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc72d1bc640 1 --2- 172.21.15.132:0/397316702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 0x55b0152679d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc7240cdbb0 tx=0x7fc72409c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ed20a74e439ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.916+0000 7fc710ff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc710ff9640 1 -- 172.21.15.132:0/397316702 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240b0e60 con 0x55b01525ab80 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc710ff9640 1 -- 172.21.15.132:0/397316702 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7fc724098a30 con 0x55b01525ab80 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc73220b740 1 -- 172.21.15.132:0/397316702 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b0151cd670 con 0x55b01525ab80 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc710ff9640 1 -- 172.21.15.132:0/397316702 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7240b5020 con 0x55b01525ab80 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc73220b740 10 client.15324 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc73220b740 1 -- 172.21.15.132:0/397316702 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b0152d81d0 con 0x55b01525ab80 2024-04-23T18:35:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc710ff9640 1 -- 172.21.15.132:0/397316702 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b0151cd670 con 0x55b01525ab80 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc710ff9640 1 -- 172.21.15.132:0/397316702 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc72406a070 con 0x55b01525ab80 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc73220b740 10 client.15324 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc73220b740 10 client.15324 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc73220b740 20 client.15324 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc73220b740 1 --2- 172.21.15.132:0/397316702 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55b0152dc8b0 0x55b0152d8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc73220b740 4 client.15324 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc73220b740 1 -- 172.21.15.132:0/397316702 --> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] -- command(tid 0: {"prefix": "status"}) -- 0x55b0151c74e0 con 0x55b0152dc8b0 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.917+0000 7fc72c9bb640 1 --2- 172.21.15.132:0/397316702 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55b0152dc8b0 0x55b0152d8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc710ff9640 10 client.15324 ms_handle_connect on v2:172.21.15.148:6836/2626530887 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc72c9bb640 1 --2- 172.21.15.132:0/397316702 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55b0152dc8b0 0x55b0152d8cf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29a281f6317b211d server_cookie=c7f3adc2fe5e60f5 in_seq=0 out_seq=0 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc710ff9640 1 -- 172.21.15.132:0/397316702 <== mds.1 v2:172.21.15.148:6836/2626530887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b0151c74e0 con 0x55b0152dc8b0 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc710ff9640 10 client.15324 handle_command_reply: tid=0 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc73220b740 1 client.15324 shutdown 2024-04-23T18:35:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc73220b740 1 -- 172.21.15.132:0/397316702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 msgr2=0x55b0152679d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc73220b740 1 --2- 172.21.15.132:0/397316702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 0x55b0152679d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc7240cdbb0 tx=0x7fc72409c3c0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc73220b740 1 -- 172.21.15.132:0/397316702 shutdown_connections 2024-04-23T18:35:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc73220b740 1 --2- 172.21.15.132:0/397316702 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55b0152dc8b0 0x55b0152d8cf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc73220b740 1 --2- 172.21.15.132:0/397316702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b01525c710 0x55b01526af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc710ff9640 0 client.15324 ms_handle_reset on v2:172.21.15.148:6836/2626530887 2024-04-23T18:35:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc73220b740 1 --2- 172.21.15.132:0/397316702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b01525bd50 0x55b015267f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc73220b740 1 --2- 172.21.15.132:0/397316702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b01525ab80 0x55b0152679d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc73220b740 1 -- 172.21.15.132:0/397316702 >> 172.21.15.132:0/397316702 conn(0x55b015256070 msgr2=0x55b0152510b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc73220b740 1 -- 172.21.15.132:0/397316702 shutdown_connections 2024-04-23T18:35:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc73220b740 1 -- 172.21.15.132:0/397316702 wait complete. 2024-04-23T18:35:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.918+0000 7fc73220b740 20 client.15324 trim_cache size 0 max 16384 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 1, 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24491, 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 111.39029087900001, 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 25, 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 89, 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 131.87794380899999 2024-04-23T18:35:54.918 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:35:54.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.919+0000 7fc72f420640 1 -- 172.21.15.132:0/2327777591 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc7140a3bb0 msgr2=0x7fc7140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.919+0000 7fc72f420640 1 --2- 172.21.15.132:0/2327777591 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc7140a3bb0 0x7fc7140a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55b01525bc50 tx=0x7fc720009780 comp rx=0 tx=0).stop 2024-04-23T18:35:54.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.919+0000 7fc72f420640 1 -- 172.21.15.132:0/2327777591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc728100e90 msgr2=0x7fc72806b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:54.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.919+0000 7fc72f420640 1 --2- 172.21.15.132:0/2327777591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc728100e90 0x7fc72806b120 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc72400d950 tx=0x7fc72400de10 comp rx=0 tx=0).stop 2024-04-23T18:35:54.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.919+0000 7fc72f420640 1 -- 172.21.15.132:0/2327777591 shutdown_connections 2024-04-23T18:35:54.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.919+0000 7fc72f420640 1 --2- 172.21.15.132:0/2327777591 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fc7140a3bb0 0x7fc7140a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.919+0000 7fc72f420640 1 --2- 172.21.15.132:0/2327777591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc728101fe0 0x7fc72819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.919+0000 7fc72f420640 1 --2- 172.21.15.132:0/2327777591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7281029a0 0x7fc72819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.919+0000 7fc72f420640 1 --2- 172.21.15.132:0/2327777591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc728100e90 0x7fc72806b120 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:54.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.919+0000 7fc72f420640 1 -- 172.21.15.132:0/2327777591 >> 172.21.15.132:0/2327777591 conn(0x7fc7280fc400 msgr2=0x7fc728105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:54.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.919+0000 7fc72f420640 1 -- 172.21.15.132:0/2327777591 shutdown_connections 2024-04-23T18:35:54.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:54.919+0000 7fc72f420640 1 -- 172.21.15.132:0/2327777591 wait complete. 2024-04-23T18:35:54.922 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T18:35:55.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:54 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3637376288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:35:55.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:54 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1258418821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:55.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:54 smithi148 ceph-mon[32311]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:55.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:54 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-23T18:35:55.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:54 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/769181272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:55.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.029+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/4117251183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b8103240 msgr2=0x7fe9b81036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.029+0000 7fe9bcbaf640 1 --2- 172.21.15.132:0/4117251183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b8103240 0x7fe9b81036c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009930 tx=0x7fe9ac02a420 comp rx=0 tx=0).stop 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.029+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/4117251183 shutdown_connections 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.029+0000 7fe9bcbaf640 1 --2- 172.21.15.132:0/4117251183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b8103c00 0x7fe9b810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.029+0000 7fe9bcbaf640 1 --2- 172.21.15.132:0/4117251183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b8103240 0x7fe9b81036c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.029+0000 7fe9bcbaf640 1 --2- 172.21.15.132:0/4117251183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b8100270 0x7fe9b8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.029+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/4117251183 >> 172.21.15.132:0/4117251183 conn(0x7fe9b80fbb80 msgr2=0x7fe9b80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.029+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/4117251183 shutdown_connections 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.029+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/4117251183 wait complete. 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9bcbaf640 1 Processor -- start 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9bcbaf640 1 -- start start 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9bcbaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b8100270 0x7fe9b81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9bcbaf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b8103240 0x7fe9b8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9bcbaf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b8103c00 0x7fe9b819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9bcbaf640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b810db10 con 0x7fe9b8103c00 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9bcbaf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b810d990 con 0x7fe9b8103240 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9bcbaf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b810dc90 con 0x7fe9b8100270 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9b6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b8103c00 0x7fe9b819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9b6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b8103c00 0x7fe9b819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60362/0 (socket says 172.21.15.132:60362) 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9b6d9d640 1 -- 172.21.15.132:0/2908802848 learned_addr learned my addr 172.21.15.132:0/2908802848 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9b5d9b640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b8103240 0x7fe9b8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9b659c640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b8100270 0x7fe9b81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9b6d9d640 1 -- 172.21.15.132:0/2908802848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b8100270 msgr2=0x7fe9b81981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9b6d9d640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b8100270 0x7fe9b81981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9b6d9d640 1 -- 172.21.15.132:0/2908802848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b8103240 msgr2=0x7fe9b8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9b6d9d640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b8103240 0x7fe9b8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9b6d9d640 1 -- 172.21.15.132:0/2908802848 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b819fe10 con 0x7fe9b8103c00 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9b659c640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b8100270 0x7fe9b81981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.030+0000 7fe9b6d9d640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b8103c00 0x7fe9b819f7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000d750 tx=0x7fe9b000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:55.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.031+0000 7fe9a77fe640 1 -- 172.21.15.132:0/2908802848 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0017070 con 0x7fe9b8103c00 2024-04-23T18:35:55.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.031+0000 7fe9a77fe640 1 -- 172.21.15.132:0/2908802848 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b00054d0 con 0x7fe9b8103c00 2024-04-23T18:35:55.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.031+0000 7fe9a77fe640 1 -- 172.21.15.132:0/2908802848 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0004c60 con 0x7fe9b8103c00 2024-04-23T18:35:55.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.031+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/2908802848 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b81a0100 con 0x7fe9b8103c00 2024-04-23T18:35:55.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.031+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/2908802848 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b81a0620 con 0x7fe9b8103c00 2024-04-23T18:35:55.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.033+0000 7fe9a77fe640 1 -- 172.21.15.132:0/2908802848 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fe9b0009bd0 con 0x7fe9b8103c00 2024-04-23T18:35:55.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.033+0000 7fe9bf99c740 1 Processor -- start 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9bf99c740 1 -- start start 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9bf99c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf7b80 0x55b7b5cf7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9bf99c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 0x55b7b5cf91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9bf99c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 0x55b7b5cfffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9bf99c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b7b5c5cc50 con 0x55b7b5cf8d50 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9bf99c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b7b5bf9830 con 0x55b7b5cf7b80 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9bf99c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b7b5c58c10 con 0x55b7b5cf9710 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9b659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 0x55b7b5cf91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9b659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 0x55b7b5cf91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60376/0 (socket says 172.21.15.132:60376) 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9b659c640 1 -- 172.21.15.132:0/471532927 learned_addr learned my addr 172.21.15.132:0/471532927 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9a77fe640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe9a00a3bb0 0x7fe9a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9b5d9b640 1 --2- 172.21.15.132:0/471532927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf7b80 0x55b7b5cf7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9b6d9d640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe9a00a3bb0 0x7fe9a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9b5d9b640 1 --2- 172.21.15.132:0/471532927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 0x55b7b5cfffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9b659c640 1 -- 172.21.15.132:0/471532927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 msgr2=0x55b7b5cfffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9b659c640 1 --2- 172.21.15.132:0/471532927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 0x55b7b5cfffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9b659c640 1 -- 172.21.15.132:0/471532927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf7b80 msgr2=0x55b7b5cf7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9b659c640 1 --2- 172.21.15.132:0/471532927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf7b80 0x55b7b5cf7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9b659c640 1 -- 172.21.15.132:0/471532927 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b596f2d0 con 0x55b7b5cf8d50 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.034+0000 7fe9a77fe640 1 -- 172.21.15.132:0/2908802848 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7fe9b00cfd50 con 0x7fe9b8103c00 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.035+0000 7fe9b659c640 1 --2- 172.21.15.132:0/471532927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 0x55b7b5cf91d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800e320 tx=0x7fe9a800e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56598ba3485866ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.035+0000 7fe9b6d9d640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe9a00a3bb0 0x7fe9a00a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00cd870 tx=0x7fe9b006aa00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.035+0000 7fe9a4ff9640 1 -- 172.21.15.132:0/471532927 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8016070 con 0x55b7b5cf8d50 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.035+0000 7fe9a4ff9640 1 -- 172.21.15.132:0/471532927 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a8010b40 con 0x55b7b5cf8d50 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.035+0000 7fe9a4ff9640 1 -- 172.21.15.132:0/471532927 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8011b70 con 0x55b7b5cf8d50 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.035+0000 7fe9bf99c740 1 -- 172.21.15.132:0/471532927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 msgr2=0x55b7b5cf91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.035+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/471532927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 0x55b7b5cf91d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800e320 tx=0x7fe9a800e7e0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.035+0000 7fe9bf99c740 1 -- 172.21.15.132:0/471532927 shutdown_connections 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.035+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/471532927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 0x55b7b5cfffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.035+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/471532927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 0x55b7b5cf91d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.035+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/471532927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf7b80 0x55b7b5cf7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.035+0000 7fe9bf99c740 1 -- 172.21.15.132:0/471532927 >> 172.21.15.132:0/471532927 conn(0x55b7b5cf3070 msgr2=0x55b7b5cf3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9bf99c740 1 -- 172.21.15.132:0/471532927 shutdown_connections 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9bf99c740 1 -- 172.21.15.132:0/471532927 wait complete. 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9bf99c740 1 -- start start 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9bf99c740 1 Processor -- start 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9bf99c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 0x55b7b5d01fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9bf99c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d024e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9bf99c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 0x55b7b5d09520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9bf99c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b7b5d00720 con 0x55b7b5cf7b80 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9bf99c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b7b5bf7470 con 0x55b7b5cf8d50 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9bf99c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b7b5d008a0 con 0x55b7b5cf9710 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9bf99c740 10 client.0 _ll_register_callbacks cb 0x55b7b5c1cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9b5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 0x55b7b5d01fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9b5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 0x55b7b5d01fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60386/0 (socket says 172.21.15.132:60386) 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9b5d9b640 1 -- 172.21.15.132:0/3073156423 learned_addr learned my addr 172.21.15.132:0/3073156423 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9b659c640 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9b5d9b640 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 0x55b7b5d09520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9b5d9b640 1 -- 172.21.15.132:0/3073156423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 msgr2=0x55b7b5d09520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9b5d9b640 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 0x55b7b5d09520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9b5d9b640 1 -- 172.21.15.132:0/3073156423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf8d50 msgr2=0x55b7b5d024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9b5d9b640 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d024e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.036+0000 7fe9b5d9b640 1 -- 172.21.15.132:0/3073156423 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b5a396b0 con 0x55b7b5cf7b80 2024-04-23T18:35:55.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe9b5d9b640 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 0x55b7b5d01fa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac02a940 tx=0x7fe9ac005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486201b21333e44c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe99dffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe99dffb640 1 -- 172.21.15.132:0/3073156423 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac035040 con 0x55b7b5cf7b80 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe99dffb640 1 -- 172.21.15.132:0/3073156423 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7fe9ac004250 con 0x55b7b5cf7b80 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3073156423 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9a8010b40 con 0x55b7b5cf7b80 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe99dffb640 1 -- 172.21.15.132:0/3073156423 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe9ac032e50 con 0x55b7b5cf7b80 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe9bf99c740 10 client.15336 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3073156423 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b7b5d0c950 con 0x55b7b5cf7b80 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe99dffb640 1 -- 172.21.15.132:0/3073156423 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a8010b40 con 0x55b7b5cf7b80 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe99dffb640 1 -- 172.21.15.132:0/3073156423 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe9ac053070 con 0x55b7b5cf7b80 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe9bf99c740 10 client.15336 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe9bf99c740 10 client.15336 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe9bf99c740 20 client.15336 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b7b5d0ef80 0x55b7b5d103f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe9bf99c740 4 client.15336 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.037+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3073156423 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7b5c6a670 con 0x55b7b5d0ef80 2024-04-23T18:35:55.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.038+0000 7fe9b659c640 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b7b5d0ef80 0x55b7b5d103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.038+0000 7fe9b659c640 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b7b5d0ef80 0x55b7b5d103f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e64f1305099accd2 server_cookie=6c1070bea673c9a8 in_seq=0 out_seq=0 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.038+0000 7fe99dffb640 10 client.15336 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe99dffb640 1 -- 172.21.15.132:0/3073156423 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7b5c6a670 con 0x55b7b5d0ef80 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe99dffb640 10 client.15336 handle_command_reply: tid=0 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe9bf99c740 1 client.15336 shutdown 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3073156423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 msgr2=0x55b7b5d01fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 0x55b7b5d01fa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac02a940 tx=0x7fe9ac005d90 comp rx=0 tx=0).stop 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3073156423 shutdown_connections 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b7b5d0ef80 0x55b7b5d103f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 0x55b7b5d09520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe99dffb640 0 client.15336 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/3073156423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 0x55b7b5d01fa0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3073156423 >> 172.21.15.132:0/3073156423 conn(0x55b7b5cf3070 msgr2=0x55b7b5cf21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:55.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3073156423 shutdown_connections 2024-04-23T18:35:55.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3073156423 wait complete. 2024-04-23T18:35:55.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.039+0000 7fe9bf99c740 20 client.15336 trim_cache size 0 max 16384 2024-04-23T18:35:55.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:54 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3637376288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:35:55.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:54 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1258418821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:35:55.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:54 smithi157 ceph-mon[32580]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:35:55.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:54 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-23T18:35:55.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:54 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/769181272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:35:55.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9bf99c740 1 Processor -- start 2024-04-23T18:35:55.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9bf99c740 1 -- start start 2024-04-23T18:35:55.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9bf99c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 0x55b7b5cfc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9bf99c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9bf99c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf9710 0x55b7b5cfcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9bf99c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b7b5d0e730 con 0x55b7b5cf7b80 2024-04-23T18:35:55.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9bf99c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b7b5d0ee00 con 0x55b7b5cf9710 2024-04-23T18:35:55.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9bf99c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b7b5d0f4d0 con 0x55b7b5cf8d50 2024-04-23T18:35:55.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9b5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 0x55b7b5cfc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9b5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 0x55b7b5cfc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60396/0 (socket says 172.21.15.132:60396) 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9b5d9b640 1 -- 172.21.15.132:0/1241624174 learned_addr learned my addr 172.21.15.132:0/1241624174 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9b5d9b640 1 -- 172.21.15.132:0/1241624174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf8d50 msgr2=0x55b7b5d094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9b5d9b640 1 --2- 172.21.15.132:0/1241624174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9b5d9b640 1 -- 172.21.15.132:0/1241624174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf9710 msgr2=0x55b7b5cfcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9b5d9b640 1 --2- 172.21.15.132:0/1241624174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf9710 0x55b7b5cfcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9b5d9b640 1 -- 172.21.15.132:0/1241624174 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b5cf87d0 con 0x55b7b5cf7b80 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe9b5d9b640 1 --2- 172.21.15.132:0/1241624174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 0x55b7b5cfc7f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac031910 tx=0x7fe9ac0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c15ed4b36c653ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.057+0000 7fe99ffff640 1 -- 172.21.15.132:0/1241624174 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac05a070 con 0x55b7b5cf7b80 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe99ffff640 1 -- 172.21.15.132:0/1241624174 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9ac050070 con 0x55b7b5cf7b80 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe99ffff640 1 -- 172.21.15.132:0/1241624174 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0035f0 con 0x55b7b5cf7b80 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe9bf99c740 1 -- 172.21.15.132:0/1241624174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 msgr2=0x55b7b5cfc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/1241624174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 0x55b7b5cfc7f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac031910 tx=0x7fe9ac0314a0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe9bf99c740 1 -- 172.21.15.132:0/1241624174 shutdown_connections 2024-04-23T18:35:55.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/1241624174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/1241624174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf9710 0x55b7b5cfcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/1241624174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf7b80 0x55b7b5cfc7f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe9bf99c740 1 -- 172.21.15.132:0/1241624174 >> 172.21.15.132:0/1241624174 conn(0x55b7b5cf3070 msgr2=0x55b7b5cee0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe9bf99c740 1 -- 172.21.15.132:0/1241624174 shutdown_connections 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe9bf99c740 1 -- 172.21.15.132:0/1241624174 wait complete. 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe9bf99c740 1 -- start start 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.058+0000 7fe9bf99c740 1 Processor -- start 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9bf99c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf7b80 0x55b7b5d046b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9bf99c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d04bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9bf99c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 0x55b7b5d07c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9bf99c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b7b5d10270 con 0x55b7b5cf8d50 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9bf99c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b7b5d0fba0 con 0x55b7b5cf7b80 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9bf99c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b7b5d10940 con 0x55b7b5cf9710 2024-04-23T18:35:55.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9bf99c740 10 client.0 _ll_register_callbacks cb 0x7fe9ac009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9b659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d04bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9b659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d04bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60412/0 (socket says 172.21.15.132:60412) 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9b659c640 1 -- 172.21.15.132:0/3808768484 learned_addr learned my addr 172.21.15.132:0/3808768484 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9b659c640 1 -- 172.21.15.132:0/3808768484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 msgr2=0x55b7b5d07c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9b659c640 1 --2- 172.21.15.132:0/3808768484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 0x55b7b5d07c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9b659c640 1 -- 172.21.15.132:0/3808768484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf7b80 msgr2=0x55b7b5d046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9b659c640 1 --2- 172.21.15.132:0/3808768484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf7b80 0x55b7b5d046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9b659c640 1 -- 172.21.15.132:0/3808768484 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b5d08200 con 0x55b7b5cf8d50 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9b659c640 1 --2- 172.21.15.132:0/3808768484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d04bf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800e2f0 tx=0x7fe9a8009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eaa72b575fa66d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe99dffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe99dffb640 1 -- 172.21.15.132:0/3808768484 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8016040 con 0x55b7b5cf8d50 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe99dffb640 1 -- 172.21.15.132:0/3808768484 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7fe9a8029430 con 0x55b7b5cf8d50 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3808768484 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9ac050070 con 0x55b7b5cf8d50 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe99dffb640 1 -- 172.21.15.132:0/3808768484 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe9a8029650 con 0x55b7b5cf8d50 2024-04-23T18:35:55.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9bf99c740 10 client.15342 fetch_fsmap learned FSMap version 25 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3808768484 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b7b5d095a0 con 0x55b7b5cf8d50 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.059+0000 7fe99dffb640 1 -- 172.21.15.132:0/3808768484 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9ac050070 con 0x55b7b5cf8d50 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.060+0000 7fe99dffb640 1 -- 172.21.15.132:0/3808768484 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe9a800f040 con 0x55b7b5cf8d50 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.060+0000 7fe9bf99c740 10 client.15342 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.060+0000 7fe9bf99c740 10 client.15342 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.060+0000 7fe9bf99c740 20 client.15342 populate_metadata read hostname 'smithi132' 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.060+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/3808768484 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b7b5d793e0 0x55b7b5d780f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.060+0000 7fe9bf99c740 4 client.15342 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.060+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3808768484 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "status"}) -- 0x55b7b5c644e0 con 0x55b7b5d793e0 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.060+0000 7fe9b5d9b640 1 --2- 172.21.15.132:0/3808768484 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b7b5d793e0 0x55b7b5d780f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.060+0000 7fe99dffb640 10 client.15342 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.060+0000 7fe9b5d9b640 1 --2- 172.21.15.132:0/3808768484 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b7b5d793e0 0x55b7b5d780f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6fddc02ccfaf15b server_cookie=714fa59582c31a30 in_seq=0 out_seq=0 2024-04-23T18:35:55.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe99dffb640 1 -- 172.21.15.132:0/3808768484 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55b7b5c644e0 con 0x55b7b5d793e0 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe99dffb640 10 client.15342 handle_command_reply: tid=0 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bf99c740 1 client.15342 shutdown 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3808768484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 msgr2=0x55b7b5d04bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/3808768484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d04bf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800e2f0 tx=0x7fe9a8009b40 comp rx=0 tx=0).stop 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3808768484 shutdown_connections 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/3808768484 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b7b5d793e0 0x55b7b5d780f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/3808768484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b5cf9710 0x55b7b5d07c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe99dffb640 0 client.15342 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/3808768484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7b5cf8d50 0x55b7b5d04bf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bf99c740 1 --2- 172.21.15.132:0/3808768484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b7b5cf7b80 0x55b7b5d046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3808768484 >> 172.21.15.132:0/3808768484 conn(0x55b7b5cf3070 msgr2=0x55b7b5cfc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3808768484 shutdown_connections 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bf99c740 1 -- 172.21.15.132:0/3808768484 wait complete. 2024-04-23T18:35:55.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bf99c740 20 client.15342 trim_cache size 0 max 16384 2024-04-23T18:35:55.060 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:35:55.060 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:35:55.060 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 0, 2024-04-23T18:35:55.060 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24515, 2024-04-23T18:35:55.060 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:35:55.060 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:35:55.060 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:35:55.060 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 113.545171397, 2024-04-23T18:35:55.060 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 25, 2024-04-23T18:35:55.060 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 89, 2024-04-23T18:35:55.061 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T18:35:55.061 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 120.540338689 2024-04-23T18:35:55.061 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:35:55.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/2908802848 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe9a00a3bb0 msgr2=0x7fe9a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bcbaf640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe9a00a3bb0 0x7fe9a00a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00cd870 tx=0x7fe9b006aa00 comp rx=0 tx=0).stop 2024-04-23T18:35:55.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/2908802848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b8103c00 msgr2=0x7fe9b819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bcbaf640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b8103c00 0x7fe9b819f7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000d750 tx=0x7fe9b000dc10 comp rx=0 tx=0).stop 2024-04-23T18:35:55.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/2908802848 shutdown_connections 2024-04-23T18:35:55.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bcbaf640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fe9a00a3bb0 0x7fe9a00a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bcbaf640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9b8103c00 0x7fe9b819f7a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bcbaf640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b8103240 0x7fe9b8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bcbaf640 1 --2- 172.21.15.132:0/2908802848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b8100270 0x7fe9b81981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/2908802848 >> 172.21.15.132:0/2908802848 conn(0x7fe9b80fbb80 msgr2=0x7fe9b8106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:55.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/2908802848 shutdown_connections 2024-04-23T18:35:55.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.061+0000 7fe9bcbaf640 1 -- 172.21.15.132:0/2908802848 wait complete. 2024-04-23T18:35:55.064 INFO:teuthology.run_tasks:Running task sequential... 2024-04-23T18:35:55.075 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-23T18:35:55.075 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-23T18:35:55.075 INFO:teuthology.task.print:Enabling mgr modules 2024-04-23T18:35:55.075 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-23T18:35:55.076 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:35:55.076 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-04-23T18:35:55.077 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-23T18:35:55.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.182+0000 7f632707b640 1 -- 172.21.15.132:0/445100439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6320100e60 msgr2=0x7f6320101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.182+0000 7f632707b640 1 --2- 172.21.15.132:0/445100439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6320100e60 0x7f6320101280 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6310009900 tx=0x7f631002a410 comp rx=0 tx=0).stop 2024-04-23T18:35:55.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.182+0000 7f632707b640 1 -- 172.21.15.132:0/445100439 shutdown_connections 2024-04-23T18:35:55.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.182+0000 7f632707b640 1 --2- 172.21.15.132:0/445100439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6320102a00 0x7f6320109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.182+0000 7f632707b640 1 --2- 172.21.15.132:0/445100439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6320102040 0x7f63201024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.182+0000 7f632707b640 1 --2- 172.21.15.132:0/445100439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6320100e60 0x7f6320101280 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.182+0000 7f632707b640 1 -- 172.21.15.132:0/445100439 >> 172.21.15.132:0/445100439 conn(0x7f63200fc440 msgr2=0x7f63200fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.182+0000 7f632707b640 1 -- 172.21.15.132:0/445100439 shutdown_connections 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.182+0000 7f632707b640 1 -- 172.21.15.132:0/445100439 wait complete. 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f632707b640 1 Processor -- start 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f632707b640 1 -- start start 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f632707b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6320100e60 0x7f6320198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f632707b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6320102040 0x7f6320198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f632707b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6320102a00 0x7f632019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f632707b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f632010db30 con 0x7f6320102a00 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f632707b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f632010d9b0 con 0x7f6320102040 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f632707b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f632010dcb0 con 0x7f6320100e60 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f6325618640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6320102a00 0x7f632019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f6325618640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6320102a00 0x7f632019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60422/0 (socket says 172.21.15.132:60422) 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f6325618640 1 -- 172.21.15.132:0/2809095862 learned_addr learned my addr 172.21.15.132:0/2809095862 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:55.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f631ffff640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6320102040 0x7f6320198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f6325618640 1 -- 172.21.15.132:0/2809095862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6320100e60 msgr2=0x7f6320198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f6324e17640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6320100e60 0x7f6320198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f6325618640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6320100e60 0x7f6320198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f6325618640 1 -- 172.21.15.132:0/2809095862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6320102040 msgr2=0x7f6320198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f6325618640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6320102040 0x7f6320198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f6325618640 1 -- 172.21.15.132:0/2809095862 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632019fd30 con 0x7f6320102a00 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f631ffff640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6320102040 0x7f6320198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.183+0000 7f6324e17640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6320100e60 0x7f6320198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.184+0000 7f6325618640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6320102a00 0x7f632019f6c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f631800eaa0 tx=0x7f631800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.184+0000 7f631dffb640 1 -- 172.21.15.132:0/2809095862 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631800cdb0 con 0x7f6320102a00 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.184+0000 7f631dffb640 1 -- 172.21.15.132:0/2809095862 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6318005020 con 0x7f6320102a00 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.184+0000 7f632707b640 1 -- 172.21.15.132:0/2809095862 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632019ffc0 con 0x7f6320102a00 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.184+0000 7f631dffb640 1 -- 172.21.15.132:0/2809095862 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63180116d0 con 0x7f6320102a00 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.184+0000 7f632707b640 1 -- 172.21.15.132:0/2809095862 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63201a04e0 con 0x7f6320102a00 2024-04-23T18:35:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.185+0000 7f632707b640 1 -- 172.21.15.132:0/2809095862 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6320108200 con 0x7f6320102a00 2024-04-23T18:35:55.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.186+0000 7f631dffb640 1 -- 172.21.15.132:0/2809095862 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f63180051d0 con 0x7f6320102a00 2024-04-23T18:35:55.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.186+0000 7f631dffb640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f63080a3c50 0x7f63080a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.186+0000 7f6324e17640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f63080a3c50 0x7f63080a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.187+0000 7f631dffb640 1 -- 172.21.15.132:0/2809095862 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7f63180d0640 con 0x7f6320102a00 2024-04-23T18:35:55.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.187+0000 7f6324e17640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f63080a3c50 0x7f63080a6110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6310006010 tx=0x7f6310000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:55.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.191+0000 7f631dffb640 1 -- 172.21.15.132:0/2809095862 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6318016030 con 0x7f6320102a00 2024-04-23T18:35:55.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.422+0000 7f632707b640 1 -- 172.21.15.132:0/2809095862 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f6320062b50 con 0x7f6320102a00 2024-04-23T18:35:55.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.565+0000 7f631dffb640 1 -- 172.21.15.132:0/2809095862 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f631809cff0 con 0x7f6320102a00 2024-04-23T18:35:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.568+0000 7f632707b640 1 -- 172.21.15.132:0/2809095862 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f63080a3c50 msgr2=0x7f63080a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.568+0000 7f632707b640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f63080a3c50 0x7f63080a6110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6310006010 tx=0x7f6310000bc0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.568+0000 7f632707b640 1 -- 172.21.15.132:0/2809095862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6320102a00 msgr2=0x7f632019f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.568+0000 7f632707b640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6320102a00 0x7f632019f6c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f631800eaa0 tx=0x7f631800ef60 comp rx=0 tx=0).stop 2024-04-23T18:35:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.568+0000 7f632707b640 1 -- 172.21.15.132:0/2809095862 shutdown_connections 2024-04-23T18:35:55.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.568+0000 7f632707b640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f63080a3c50 0x7f63080a6110 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.568+0000 7f632707b640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6320102a00 0x7f632019f6c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.568+0000 7f632707b640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6320102040 0x7f6320198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.568+0000 7f632707b640 1 --2- 172.21.15.132:0/2809095862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6320100e60 0x7f6320198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.568+0000 7f632707b640 1 -- 172.21.15.132:0/2809095862 >> 172.21.15.132:0/2809095862 conn(0x7f63200fc440 msgr2=0x7f63201053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:55.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.568+0000 7f632707b640 1 -- 172.21.15.132:0/2809095862 shutdown_connections 2024-04-23T18:35:55.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.568+0000 7f632707b640 1 -- 172.21.15.132:0/2809095862 wait complete. 2024-04-23T18:35:55.578 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-23T18:35:55.618 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:55 smithi132 ceph-mon[29178]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-23T18:35:55.618 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:55 smithi132 ceph-mon[29178]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T18:35:55.618 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:55 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2809095862' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.695+0000 7f2423e64640 1 -- 172.21.15.132:0/2426436964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f241c100660 msgr2=0x7f241c100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.695+0000 7f2423e64640 1 --2- 172.21.15.132:0/2426436964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f241c100660 0x7f241c100a80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f240c00aa10 tx=0x7f240c02b720 comp rx=0 tx=0).stop 2024-04-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.695+0000 7f2423e64640 1 -- 172.21.15.132:0/2426436964 shutdown_connections 2024-04-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.695+0000 7f2423e64640 1 --2- 172.21.15.132:0/2426436964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c102210 0x7f241c108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.695+0000 7f2423e64640 1 --2- 172.21.15.132:0/2426436964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c101850 0x7f241c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.695+0000 7f2423e64640 1 --2- 172.21.15.132:0/2426436964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f241c100660 0x7f241c100a80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.695+0000 7f2423e64640 1 -- 172.21.15.132:0/2426436964 >> 172.21.15.132:0/2426436964 conn(0x7f241c0fbbf0 msgr2=0x7f241c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.695+0000 7f2423e64640 1 -- 172.21.15.132:0/2426436964 shutdown_connections 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.695+0000 7f2423e64640 1 -- 172.21.15.132:0/2426436964 wait complete. 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2423e64640 1 Processor -- start 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2423e64640 1 -- start start 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2423e64640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f241c100660 0x7f241c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2423e64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c101850 0x7f241c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2423e64640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c102210 0x7f241c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2423e64640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10b920 con 0x7f241c100660 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2423e64640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10b7a0 con 0x7f241c102210 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2423e64640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10baa0 con 0x7f241c101850 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2421c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f241c100660 0x7f241c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2421c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f241c100660 0x7f241c1981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60474/0 (socket says 172.21.15.132:60474) 2024-04-23T18:35:55.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2421c00640 1 -- 172.21.15.132:0/838986932 learned_addr learned my addr 172.21.15.132:0/838986932 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2422401640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c102210 0x7f241c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f24213ff640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c101850 0x7f241c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2422401640 1 -- 172.21.15.132:0/838986932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c101850 msgr2=0x7f241c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2422401640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c101850 0x7f241c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2422401640 1 -- 172.21.15.132:0/838986932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f241c100660 msgr2=0x7f241c1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2422401640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f241c100660 0x7f241c1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2422401640 1 -- 172.21.15.132:0/838986932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c19fdb0 con 0x7f241c102210 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2421c00640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f241c100660 0x7f241c1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f24213ff640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c101850 0x7f241c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.696+0000 7f2422401640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c102210 0x7f241c19f740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2414009fd0 tx=0x7f241400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.697+0000 7f2412ffd640 1 -- 172.21.15.132:0/838986932 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400ce20 con 0x7f241c102210 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.697+0000 7f2412ffd640 1 -- 172.21.15.132:0/838986932 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2414005020 con 0x7f241c102210 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.697+0000 7f2412ffd640 1 -- 172.21.15.132:0/838986932 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414010690 con 0x7f241c102210 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.697+0000 7f2423e64640 1 -- 172.21.15.132:0/838986932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c1a00a0 con 0x7f241c102210 2024-04-23T18:35:55.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.697+0000 7f2423e64640 1 -- 172.21.15.132:0/838986932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c1a05f0 con 0x7f241c102210 2024-04-23T18:35:55.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.698+0000 7f2423e64640 1 -- 172.21.15.132:0/838986932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241c1a08b0 con 0x7f241c102210 2024-04-23T18:35:55.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.699+0000 7f2412ffd640 1 -- 172.21.15.132:0/838986932 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f24140054d0 con 0x7f241c102210 2024-04-23T18:35:55.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.699+0000 7f2412ffd640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24080a3ca0 0x7f24080a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:55.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.699+0000 7f2421c00640 1 -- 172.21.15.132:0/838986932 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24080a3ca0 msgr2=0x7f24080a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/636352941 2024-04-23T18:35:55.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.700+0000 7f2412ffd640 1 -- 172.21.15.132:0/838986932 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f24140d0bc0 con 0x7f241c102210 2024-04-23T18:35:55.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.700+0000 7f2421c00640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24080a3ca0 0x7f24080a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:35:55.696 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.704+0000 7f2412ffd640 1 -- 172.21.15.132:0/838986932 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f241409d510 con 0x7f241c102210 2024-04-23T18:35:55.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.894+0000 7f2423e64640 1 -- 172.21.15.132:0/838986932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f241c1a0f30 con 0x7f241c102210 2024-04-23T18:35:55.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.900+0000 7f2412ffd640 1 -- 172.21.15.132:0/838986932 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v26)=0 v26) ==== 138+0+0 (secure 0 0 0) 0x7f241409ce10 con 0x7f241c102210 2024-04-23T18:35:55.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.901+0000 7f2421c00640 1 -- 172.21.15.132:0/838986932 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24080a3ca0 msgr2=0x7f24080a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/636352941 2024-04-23T18:35:55.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.901+0000 7f2421c00640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24080a3ca0 0x7f24080a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:35:55.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.902+0000 7f2423e64640 1 -- 172.21.15.132:0/838986932 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24080a3ca0 msgr2=0x7f24080a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:35:55.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.902+0000 7f2423e64640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24080a3ca0 0x7f24080a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.902+0000 7f2423e64640 1 -- 172.21.15.132:0/838986932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c102210 msgr2=0x7f241c19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:55.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.902+0000 7f2423e64640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c102210 0x7f241c19f740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2414009fd0 tx=0x7f241400ef10 comp rx=0 tx=0).stop 2024-04-23T18:35:55.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.902+0000 7f2423e64640 1 -- 172.21.15.132:0/838986932 shutdown_connections 2024-04-23T18:35:55.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.902+0000 7f2423e64640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f24080a3ca0 0x7f24080a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.902+0000 7f2423e64640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c101850 0x7f241c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.902+0000 7f2423e64640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c102210 0x7f241c19f740 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.902+0000 7f2423e64640 1 --2- 172.21.15.132:0/838986932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f241c100660 0x7f241c1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:55.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.902+0000 7f2423e64640 1 -- 172.21.15.132:0/838986932 >> 172.21.15.132:0/838986932 conn(0x7f241c0fbbf0 msgr2=0x7f241c106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:55.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.902+0000 7f2423e64640 1 -- 172.21.15.132:0/838986932 shutdown_connections 2024-04-23T18:35:55.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:55.902+0000 7f2423e64640 1 -- 172.21.15.132:0/838986932 wait complete. 2024-04-23T18:35:55.905 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:55 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: ignoring --setuser ceph since I am not root 2024-04-23T18:35:55.905 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:55 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: ignoring --setgroup ceph since I am not root 2024-04-23T18:35:55.905 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:55 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:55.760+0000 7fa7b6a36200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:35:55.906 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-23T18:35:55.921 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:55 smithi157 ceph-mon[32580]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-23T18:35:55.921 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:55 smithi157 ceph-mon[32580]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T18:35:55.922 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:55 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2809095862' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T18:35:55.922 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:55 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: ignoring --setuser ceph since I am not root 2024-04-23T18:35:55.922 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:55 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: ignoring --setgroup ceph since I am not root 2024-04-23T18:35:55.922 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:55 smithi157 ceph-mgr[34312]: -- 172.21.15.157:0/1155184124 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b02d0fc5a0 con 0x55b02d0d0c00 2024-04-23T18:35:55.922 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:55 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:55.746+0000 7fe7957da200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:35:55.922 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:55 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:55.928+0000 7fe7957da200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:35:55.926 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:55 smithi148 ceph-mon[32311]: from='mgr.14152 172.21.15.132:0/4185502617' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-23T18:35:55.926 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:55 smithi148 ceph-mon[32311]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T18:35:55.926 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:55 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2809095862' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T18:35:55.927 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: ignoring --setuser ceph since I am not root 2024-04-23T18:35:55.927 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: ignoring --setgroup ceph since I am not root 2024-04-23T18:35:55.927 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:55 smithi148 ceph-mgr[33699]: -- 172.21.15.148:0/4129358822 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5585e306e5a0 con 0x5585e3042c00 2024-04-23T18:35:55.927 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:55.750+0000 7f132814a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:35:55.927 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:55.934+0000 7f132814a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:35:56.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.058+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/8321148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ee41009d0 msgr2=0x7f1ee410a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:56.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.058+0000 7f1ee9cf6640 1 --2- 172.21.15.132:0/8321148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ee41009d0 0x7f1ee410a580 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009ca0 tx=0x7f1edc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:35:56.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.058+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/8321148 shutdown_connections 2024-04-23T18:35:56.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.058+0000 7f1ee9cf6640 1 --2- 172.21.15.132:0/8321148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ee41009d0 0x7f1ee410a580 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:56.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.058+0000 7f1ee9cf6640 1 --2- 172.21.15.132:0/8321148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee4100010 0x7f1ee4100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:56.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.058+0000 7f1ee9cf6640 1 --2- 172.21.15.132:0/8321148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee41035a0 0x7f1ee41039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:56.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.058+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/8321148 >> 172.21.15.132:0/8321148 conn(0x7f1ee40fbbb0 msgr2=0x7f1ee40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:56.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.058+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/8321148 shutdown_connections 2024-04-23T18:35:56.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.058+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/8321148 wait complete. 2024-04-23T18:35:56.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.059+0000 7f1ee9cf6640 1 Processor -- start 2024-04-23T18:35:56.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.059+0000 7f1ee9cf6640 1 -- start start 2024-04-23T18:35:56.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.059+0000 7f1ee9cf6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee4100010 0x7f1ee4102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:56.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.059+0000 7f1ee9cf6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee41009d0 0x7f1ee41012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:56.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.059+0000 7f1ee9cf6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ee41035a0 0x7f1ee4101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:56.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.059+0000 7f1ee9cf6640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee410db30 con 0x7f1ee41035a0 2024-04-23T18:35:56.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.059+0000 7f1ee9cf6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee410d9b0 con 0x7f1ee4100010 2024-04-23T18:35:56.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.059+0000 7f1ee9cf6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee410dcb0 con 0x7f1ee41009d0 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.059+0000 7f1ee37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee4100010 0x7f1ee4102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.059+0000 7f1ee3fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ee41035a0 0x7f1ee4101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.059+0000 7f1ee2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee41009d0 0x7f1ee41012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.059+0000 7f1ee37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee4100010 0x7f1ee4102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.132:51256/0 (socket says 172.21.15.132:51256) 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee37fe640 1 -- 172.21.15.132:0/342565171 learned_addr learned my addr 172.21.15.132:0/342565171 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee37fe640 1 -- 172.21.15.132:0/342565171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee41009d0 msgr2=0x7f1ee41012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee37fe640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee41009d0 0x7f1ee41012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee37fe640 1 -- 172.21.15.132:0/342565171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ee41035a0 msgr2=0x7f1ee4101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee37fe640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ee41035a0 0x7f1ee4101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee37fe640 1 -- 172.21.15.132:0/342565171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee41020b0 con 0x7f1ee4100010 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee2ffd640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee41009d0 0x7f1ee41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee37fe640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee4100010 0x7f1ee4102ca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400e9c0 tx=0x7f1ed400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee0ff9640 1 -- 172.21.15.132:0/342565171 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed400cd80 con 0x7f1ee4100010 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/342565171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee41a2210 con 0x7f1ee4100010 2024-04-23T18:35:56.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee0ff9640 1 -- 172.21.15.132:0/342565171 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ed40035f0 con 0x7f1ee4100010 2024-04-23T18:35:56.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee0ff9640 1 -- 172.21.15.132:0/342565171 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4010690 con 0x7f1ee4100010 2024-04-23T18:35:56.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.060+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/342565171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee41a2760 con 0x7f1ee4100010 2024-04-23T18:35:56.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.062+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/342565171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee4109460 con 0x7f1ee4100010 2024-04-23T18:35:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.063+0000 7f1ee0ff9640 1 -- 172.21.15.132:0/342565171 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f1ed4004000 con 0x7f1ee4100010 2024-04-23T18:35:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.063+0000 7f1ee0ff9640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ecc0a3d20 0x7f1ecc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.063+0000 7f1ee2ffd640 1 -- 172.21.15.132:0/342565171 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ecc0a3d20 msgr2=0x7f1ecc0a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/636352941 2024-04-23T18:35:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.063+0000 7f1ee0ff9640 1 -- 172.21.15.132:0/342565171 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f1ed40d0970 con 0x7f1ee4100010 2024-04-23T18:35:56.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.063+0000 7f1ee2ffd640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ecc0a3d20 0x7f1ecc0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:35:56.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.068+0000 7f1ee0ff9640 1 -- 172.21.15.132:0/342565171 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ed409d320 con 0x7f1ee4100010 2024-04-23T18:35:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.264+0000 7f1ee2ffd640 1 -- 172.21.15.132:0/342565171 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ecc0a3d20 msgr2=0x7f1ecc0a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/636352941 2024-04-23T18:35:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.264+0000 7f1ee2ffd640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ecc0a3d20 0x7f1ecc0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:35:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.265+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/342565171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f1ee4062b50 con 0x7f1ee4100010 2024-04-23T18:35:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.270+0000 7f1ee0ff9640 1 -- 172.21.15.132:0/342565171 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v27)=0 v27) ==== 133+0+0 (secure 0 0 0) 0x7f1ed409cc20 con 0x7f1ee4100010 2024-04-23T18:35:56.264 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:56 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:56.037+0000 7f132814a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:35:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.273+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/342565171 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ecc0a3d20 msgr2=0x7f1ecc0a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:35:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.273+0000 7f1ee9cf6640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ecc0a3d20 0x7f1ecc0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.273+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/342565171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee4100010 msgr2=0x7f1ee4102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.273+0000 7f1ee9cf6640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee4100010 0x7f1ee4102ca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400e9c0 tx=0x7f1ed400ee80 comp rx=0 tx=0).stop 2024-04-23T18:35:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.273+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/342565171 shutdown_connections 2024-04-23T18:35:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.273+0000 7f1ee9cf6640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1ecc0a3d20 0x7f1ecc0a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.273+0000 7f1ee9cf6640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee41009d0 0x7f1ee41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.273+0000 7f1ee9cf6640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ee41035a0 0x7f1ee4101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.273+0000 7f1ee9cf6640 1 --2- 172.21.15.132:0/342565171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee4100010 0x7f1ee4102ca0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.273+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/342565171 >> 172.21.15.132:0/342565171 conn(0x7f1ee40fbbb0 msgr2=0x7f1ee4106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.274+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/342565171 shutdown_connections 2024-04-23T18:35:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:56.274+0000 7f1ee9cf6640 1 -- 172.21.15.132:0/342565171 wait complete. 2024-04-23T18:35:56.279 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:55 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:55.956+0000 7fa7b6a36200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:35:56.279 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:56 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:56.068+0000 7fa7b6a36200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:35:56.280 INFO:teuthology.run_tasks:Running task kclient... 2024-04-23T18:35:56.305 INFO:tasks.kclient:Mounting kernel clients... 2024-04-23T18:35:56.305 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-23T18:35:56.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:35:56.305 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-23T18:35:56.306 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:56 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:56.032+0000 7fe7957da200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:35:56.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:35:56.321 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-04-23T18:35:56.386 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2024-04-23T18:35:56.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:35:56.388 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-23T18:35:56.388 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T18:35:56.389 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T18:35:56.389 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-23T18:35:56.555 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:56 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:56.393+0000 7fa7b6a36200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:35:56.556 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:56 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:56.482+0000 7fa7b6a36200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:35:56.686 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:35:56.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:56 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2809095862' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T18:35:56.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:56 smithi148 ceph-mon[32311]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T18:35:56.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:56 smithi148 ceph-mon[32311]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-23T18:35:56.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:56 smithi148 ceph-mon[32311]: from='client.? ' entity='client.admin' 2024-04-23T18:35:56.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:56 smithi148 ceph-mon[32311]: from='client.? ' entity='client.admin' 2024-04-23T18:35:56.764 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:56 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:56.347+0000 7f132814a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:35:56.764 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:56 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:56.432+0000 7f132814a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:56 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2809095862' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:56 smithi157 ceph-mon[32580]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:56 smithi157 ceph-mon[32580]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-23T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:56 smithi157 ceph-mon[32580]: from='client.? ' entity='client.admin' 2024-04-23T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:56 smithi157 ceph-mon[32580]: from='client.? ' entity='client.admin' 2024-04-23T18:35:56.790 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:56 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:56.340+0000 7fe7957da200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:35:56.790 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:56 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:56.425+0000 7fe7957da200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:35:56.808 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:56 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2809095862' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T18:35:56.809 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:56 smithi132 ceph-mon[29178]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T18:35:56.809 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:56 smithi132 ceph-mon[29178]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-23T18:35:56.809 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:56 smithi132 ceph-mon[29178]: from='client.? ' entity='client.admin' 2024-04-23T18:35:56.809 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:56 smithi132 ceph-mon[29178]: from='client.? ' entity='client.admin' 2024-04-23T18:35:57.941 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:57 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:57.606+0000 7fe7957da200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:35:57.942 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:57 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:57.777+0000 7fe7957da200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:35:57.942 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:57 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:57.948+0000 7fe7957da200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:35:57.955 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:57 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:57.622+0000 7f132814a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:35:57.955 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:57 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:57.792+0000 7f132814a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:35:57.955 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:57 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:57.962+0000 7f132814a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:35:57.960 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:57 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:57.708+0000 7fa7b6a36200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:35:57.960 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:57 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:57.882+0000 7fa7b6a36200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:35:58.269 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:58.066+0000 7fa7b6a36200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:35:58.514 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:58 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:58.250+0000 7f132814a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:35:58.514 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:58 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:58.342+0000 7f132814a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:35:58.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.532+0000 7f1d539e2700 1 -- 172.21.15.132:0/2221799792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d4c1023a0 msgr2=0x7f1d4c102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:58.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.532+0000 7f1d539e2700 1 --2- 172.21.15.132:0/2221799792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d4c1023a0 0x7f1d4c102780 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0099c0 tx=0x7f1d3c009cd0 comp rx=0 tx=0).stop 2024-04-23T18:35:58.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.532+0000 7f1d539e2700 1 -- 172.21.15.132:0/2221799792 shutdown_connections 2024-04-23T18:35:58.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.532+0000 7f1d539e2700 1 --2- 172.21.15.132:0/2221799792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4c106cc0 0x7f1d4c10e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:58.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.532+0000 7f1d539e2700 1 --2- 172.21.15.132:0/2221799792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d4c1023a0 0x7f1d4c102780 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:58.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.532+0000 7f1d539e2700 1 --2- 172.21.15.132:0/2221799792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d4c102d50 0x7f1d4c1065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:58.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.532+0000 7f1d539e2700 1 -- 172.21.15.132:0/2221799792 >> 172.21.15.132:0/2221799792 conn(0x7f1d4c077fb0 msgr2=0x7f1d4c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:58.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.533+0000 7f1d539e2700 1 -- 172.21.15.132:0/2221799792 shutdown_connections 2024-04-23T18:35:58.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.533+0000 7f1d539e2700 1 -- 172.21.15.132:0/2221799792 wait complete. 2024-04-23T18:35:58.526 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:58.367+0000 7fa7b6a36200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:35:58.526 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:58.461+0000 7fa7b6a36200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:35:58.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.533+0000 7f1d539e2700 1 Processor -- start 2024-04-23T18:35:58.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.533+0000 7f1d539e2700 1 -- start start 2024-04-23T18:35:58.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d539e2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d4c1023a0 0x7f1d4c199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:58.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d539e2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d4c102d50 0x7f1d4c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:58.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d539e2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4c106cc0 0x7f1d4c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:58.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d539e2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c111bb0 con 0x7f1d4c102d50 2024-04-23T18:35:58.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d539e2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c111a30 con 0x7f1d4c1023a0 2024-04-23T18:35:58.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d539e2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c111d30 con 0x7f1d4c106cc0 2024-04-23T18:35:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d50f7d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d4c102d50 0x7f1d4c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d50f7d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d4c102d50 0x7f1d4c199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60510/0 (socket says 172.21.15.132:60510) 2024-04-23T18:35:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d50f7d700 1 -- 172.21.15.132:0/281244320 learned_addr learned my addr 172.21.15.132:0/281244320 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:35:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d5177e700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d4c1023a0 0x7f1d4c199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d51f7f700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4c106cc0 0x7f1d4c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d50f7d700 1 -- 172.21.15.132:0/281244320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4c106cc0 msgr2=0x7f1d4c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d50f7d700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4c106cc0 0x7f1d4c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d50f7d700 1 -- 172.21.15.132:0/281244320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d4c1023a0 msgr2=0x7f1d4c199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d50f7d700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d4c1023a0 0x7f1d4c199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d50f7d700 1 -- 172.21.15.132:0/281244320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c19e210 con 0x7f1d4c102d50 2024-04-23T18:35:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d51f7f700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4c106cc0 0x7f1d4c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:35:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d5177e700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d4c1023a0 0x7f1d4c199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:35:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.534+0000 7f1d50f7d700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d4c102d50 0x7f1d4c199790 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800e970 tx=0x7f1d4800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.535+0000 7f1d427fc700 1 -- 172.21.15.132:0/281244320 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48004a30 con 0x7f1d4c102d50 2024-04-23T18:35:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.535+0000 7f1d539e2700 1 -- 172.21.15.132:0/281244320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4c19e500 con 0x7f1d4c102d50 2024-04-23T18:35:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.535+0000 7f1d427fc700 1 -- 172.21.15.132:0/281244320 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d4800cc90 con 0x7f1d4c102d50 2024-04-23T18:35:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.535+0000 7f1d427fc700 1 -- 172.21.15.132:0/281244320 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48005560 con 0x7f1d4c102d50 2024-04-23T18:35:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.535+0000 7f1d539e2700 1 -- 172.21.15.132:0/281244320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4c0fd350 con 0x7f1d4c102d50 2024-04-23T18:35:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.536+0000 7f1d539e2700 1 -- 172.21.15.132:0/281244320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4c103150 con 0x7f1d4c102d50 2024-04-23T18:35:58.531 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:58 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:58.238+0000 7fe7957da200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:35:58.531 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:58 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:58.330+0000 7fe7957da200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:35:58.531 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:58 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:58.534+0000 7fe7957da200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:35:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.537+0000 7f1d427fc700 1 -- 172.21.15.132:0/281244320 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f1d48004bd0 con 0x7f1d4c102d50 2024-04-23T18:35:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.537+0000 7f1d427fc700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1d380a3c90 0x7f1d380a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.537+0000 7f1d5177e700 1 -- 172.21.15.132:0/281244320 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1d380a3c90 msgr2=0x7f1d380a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/636352941 2024-04-23T18:35:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.538+0000 7f1d427fc700 1 -- 172.21.15.132:0/281244320 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f1d480d05b0 con 0x7f1d4c102d50 2024-04-23T18:35:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.538+0000 7f1d5177e700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1d380a3c90 0x7f1d380a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:35:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.542+0000 7f1d427fc700 1 -- 172.21.15.132:0/281244320 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d48005700 con 0x7f1d4c102d50 2024-04-23T18:35:58.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.739+0000 7f1d5177e700 1 -- 172.21.15.132:0/281244320 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1d380a3c90 msgr2=0x7f1d380a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/636352941 2024-04-23T18:35:58.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.739+0000 7f1d5177e700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1d380a3c90 0x7f1d380a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:35:58.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.795+0000 7f1d539e2700 1 -- 172.21.15.132:0/281244320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1d4c0fd670 con 0x7f1d4c102d50 2024-04-23T18:35:58.788 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:58.674+0000 7fa7b6a36200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:35:58.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.796+0000 7f1d427fc700 1 -- 172.21.15.132:0/281244320 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f1d4809cf60 con 0x7f1d4c102d50 2024-04-23T18:35:58.788 INFO:teuthology.orchestra.run.smithi132.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T18:35:58.789 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:58 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:58.729+0000 7fe7957da200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:35:58.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.798+0000 7f1d539e2700 1 -- 172.21.15.132:0/281244320 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1d380a3c90 msgr2=0x7f1d380a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:35:58.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.798+0000 7f1d539e2700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1d380a3c90 0x7f1d380a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:58.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.798+0000 7f1d539e2700 1 -- 172.21.15.132:0/281244320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d4c102d50 msgr2=0x7f1d4c199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:58.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.798+0000 7f1d539e2700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d4c102d50 0x7f1d4c199790 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800e970 tx=0x7f1d4800ec80 comp rx=0 tx=0).stop 2024-04-23T18:35:58.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.798+0000 7f1d539e2700 1 -- 172.21.15.132:0/281244320 shutdown_connections 2024-04-23T18:35:58.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.798+0000 7f1d539e2700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d4c106cc0 0x7f1d4c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:58.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.798+0000 7f1d539e2700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d4c1023a0 0x7f1d4c199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:58.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.798+0000 7f1d539e2700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f1d380a3c90 0x7f1d380a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:58.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.798+0000 7f1d539e2700 1 --2- 172.21.15.132:0/281244320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d4c102d50 0x7f1d4c199790 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:58.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.798+0000 7f1d539e2700 1 -- 172.21.15.132:0/281244320 >> 172.21.15.132:0/281244320 conn(0x7f1d4c077fb0 msgr2=0x7f1d4c1055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:58.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.798+0000 7f1d539e2700 1 -- 172.21.15.132:0/281244320 shutdown_connections 2024-04-23T18:35:58.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:58.798+0000 7f1d539e2700 1 -- 172.21.15.132:0/281244320 wait complete. 2024-04-23T18:35:58.839 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:58 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:58.547+0000 7f132814a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:35:58.839 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:58 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:58.742+0000 7f132814a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:35:59.110 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:58.879+0000 7fa7b6a36200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:35:59.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:35:58 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/281244320' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T18:35:59.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:35:58 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/281244320' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T18:35:59.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:35:58 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/281244320' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T18:35:59.507 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T18:35:59.507 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T18:35:59.507 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2024-04-23T18:35:59.507 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T18:35:59.507 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T18:35:59.507 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T18:35:59.507 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T18:35:59.507 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T18:35:59.508 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T18:35:59.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:35:59.508 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2024-04-23T18:35:59.523 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T18:35:59.523 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T18:35:59.524 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2024-04-23T18:35:59.525 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-23T18:35:59.525 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42001sec preferred_lft 42001sec 2024-04-23T18:35:59.525 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::502d:ac94:fffc:d962/64 scope link noprefixroute 2024-04-23T18:35:59.525 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-23T18:35:59.525 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T18:35:59.525 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:35:59.525 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-23T18:35:59.525 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add name ceph-brx type bridge 2024-04-23T18:35:59.526 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr flush dev ceph-brx 2024-04-23T18:35:59.526 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set ceph-brx up 2024-04-23T18:35:59.526 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T18:35:59.526 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-23T18:35:59.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:35:59.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:35:59.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:35:59.717 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T18:35:59.738 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:59 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:59.488+0000 7fa7b6a36200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:35:59.740 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-04-23T18:35:59.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:35:59.743 DEBUG:teuthology.orchestra.run.smithi132:> route 2024-04-23T18:35:59.745 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:59 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:59.332+0000 7fe7957da200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:35:59.758 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:59 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:59.346+0000 7f132814a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:35:59.811 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2024-04-23T18:35:59.811 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T18:35:59.811 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T18:35:59.811 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T18:35:59.811 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T18:35:59.812 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:35:59.812 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-23T18:35:59.812 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T18:35:59.812 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T18:35:59.812 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T18:35:59.812 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-23T18:35:59.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:35:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:36:00.014 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:59 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:59.766+0000 7f132814a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:36:00.014 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:35:59 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:35:59.862+0000 7f132814a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:36:00.020 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:35:59 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:35:59.946+0000 7fa7b6a36200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:36:00.039 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:59 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:59.752+0000 7fe7957da200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:36:00.039 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:35:59 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:35:59.849+0000 7fe7957da200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:36:00.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:36:00.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:36:00.105 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-23T18:36:00.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:36:00.119 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2024-04-23T18:36:00.174 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:36:00.174 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-23T18:36:00.174 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T18:36:00.174 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-23T18:36:00.174 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-23T18:36:00.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:36:00.270 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:00.049+0000 7fa7b6a36200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:36:00.271 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:00.265+0000 7fa7b6a36200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:36:00.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:36:00.291 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-23T18:36:00.292 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:36:00.292 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-23T18:36:00.292 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-23T18:36:00.292 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T18:36:00.292 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-23T18:36:00.292 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-23T18:36:00.292 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-23T18:36:00.292 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-23T18:36:00.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:36:00.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:36:00.416 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:36:00.416 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-23T18:36:00.416 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.0 up 2024-04-23T18:36:00.416 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T18:36:00.416 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-23T18:36:00.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:36:00.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:36:00.497 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T18:36:00.497 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T18:36:00.511 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T18:36:00.511 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T18:36:00.512 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T18:36:00.514 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:36:00 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:36:00.062+0000 7f132814a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:36:00.539 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:36:00 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:36:00.050+0000 7fe7957da200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:36:00.566 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-23T18:36:00.566 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-23T18:36:00.566 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/047319ef-c4f2-44a4-baf2-edbcd2a1c750 2024-04-23T18:36:00.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:36:00.566 DEBUG:teuthology.orchestra.run.smithi132:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/047319ef-c4f2-44a4-baf2-edbcd2a1c750 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-23T18:36:00.658 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-23T18:36:00.659 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-23T18:36:00.659 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2024-04-23T18:36:00.765 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-23T18:36:00.765 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-23T18:36:00.765 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2024-04-23T18:36:00.765 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.132:3300,172.21.15.148:3300,172.21.15.157:3300" 2024-04-23T18:36:00.765 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.132:3300,172.21.15.148:3300,172.21.15.157:3300:/volumes/qa/sv_1/047319ef-c4f2-44a4-baf2-edbcd2a1c750 2024-04-23T18:36:00.765 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=c34642b4-019e-11ef-bc93-c7b262605968" will pass to kernel 2024-04-23T18:36:00.772 INFO:teuthology.orchestra.run.smithi132.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-23T18:36:00.772 INFO:teuthology.orchestra.run.smithi132.stdout: You just mounted a file system that supports labels which does not 2024-04-23T18:36:00.772 INFO:teuthology.orchestra.run.smithi132.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-23T18:36:00.772 INFO:teuthology.orchestra.run.smithi132.stdout: applications will generate AVC messages and not be allowed access to 2024-04-23T18:36:00.772 INFO:teuthology.orchestra.run.smithi132.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-23T18:36:00.773 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-23T18:36:00.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:36:00.773 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T18:36:00.801 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T18:36:00.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T18:36:00.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:36:00.898 INFO:teuthology.orchestra.run.smithi132.stdout:client24974 2024-04-23T18:36:00.898 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:36:00.898 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T18:36:00.906 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:36:00 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:36:00.577+0000 7fe7957da200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:36:00.906 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:36:00 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:36:00.670+0000 7fe7957da200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:36:00.915 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T18:36:00.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:36:00 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:36:00.589+0000 7f132814a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:36:00.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:36:00 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:36:00.680+0000 7f132814a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:36:00.923 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:00.830+0000 7fa7b6a36200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:36:00.923 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:00.930+0000 7fa7b6a36200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:36:00.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T18:36:00.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:36:00.977 INFO:teuthology.orchestra.run.smithi132.stdout:client24974 2024-04-23T18:36:00.978 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T18:36:01.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T18:36:01.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:36:01.034 INFO:teuthology.orchestra.run.smithi132.stdout:client24974 2024-04-23T18:36:01.034 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c34642b4-019e-11ef-bc93-c7b262605968.client24974/status') 2024-04-23T18:36:01.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:36:01.092 INFO:teuthology.orchestra.run.smithi132.stdout:instance: client.24974 (3)172.21.15.132:0/3851789271 2024-04-23T18:36:01.092 INFO:teuthology.orchestra.run.smithi132.stdout:blocklisted: false 2024-04-23T18:36:01.092 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records in 2024-04-23T18:36:01.092 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records out 2024-04-23T18:36:01.092 INFO:teuthology.orchestra.run.smithi132.stderr:72 bytes copied, 2.392e-05 s, 3.0 MB/s 2024-04-23T18:36:01.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:36:01.100 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T18:36:01.109 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:36:01.109 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-04-23T18:36:01.109 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-23T18:36:01.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.219+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2949671050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f08100290 msgr2=0x7f4f081006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.219+0000 7f4f0f9e8640 1 --2- 172.21.15.132:0/2949671050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f08100290 0x7f4f081006b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009930 tx=0x7f4f0402a420 comp rx=0 tx=0).stop 2024-04-23T18:36:01.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.219+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2949671050 shutdown_connections 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.219+0000 7f4f0f9e8640 1 --2- 172.21.15.132:0/2949671050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f08103260 0x7f4f081036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.219+0000 7f4f0f9e8640 1 --2- 172.21.15.132:0/2949671050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f08103c20 0x7f4f0810a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.219+0000 7f4f0f9e8640 1 --2- 172.21.15.132:0/2949671050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f08100290 0x7f4f081006b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.219+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2949671050 >> 172.21.15.132:0/2949671050 conn(0x7f4f080fbbc0 msgr2=0x7f4f080fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.219+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2949671050 shutdown_connections 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.219+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2949671050 wait complete. 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0f9e8640 1 Processor -- start 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0f9e8640 1 -- start start 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0f9e8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f08100290 0x7f4f08198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0f9e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f08103260 0x7f4f081986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0f9e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f08103c20 0x7f4f0819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0f9e8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0810daf0 con 0x7f4f08100290 2024-04-23T18:36:01.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0f9e8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0810d970 con 0x7f4f08103260 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0f9e8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0810dc70 con 0x7f4f08103c20 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0d784640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f08100290 0x7f4f08198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0d784640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f08100290 0x7f4f08198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57454/0 (socket says 172.21.15.132:57454) 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0d784640 1 -- 172.21.15.132:0/2275837438 learned_addr learned my addr 172.21.15.132:0/2275837438 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0df85640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f08103c20 0x7f4f0819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.220+0000 7f4f0cf83640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f08103260 0x7f4f081986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.221+0000 7f4f0df85640 1 -- 172.21.15.132:0/2275837438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f08103260 msgr2=0x7f4f081986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.221+0000 7f4f0df85640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f08103260 0x7f4f081986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.221+0000 7f4f0df85640 1 -- 172.21.15.132:0/2275837438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f08100290 msgr2=0x7f4f08198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.221+0000 7f4f0df85640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f08100290 0x7f4f08198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.221+0000 7f4f0df85640 1 -- 172.21.15.132:0/2275837438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0819fdb0 con 0x7f4f08103c20 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.221+0000 7f4f0d784640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f08100290 0x7f4f08198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.221+0000 7f4f0df85640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f08103c20 0x7f4f0819f740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009fd0 tx=0x7f4f0000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.221+0000 7f4efe7fc640 1 -- 172.21.15.132:0/2275837438 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000ce20 con 0x7f4f08103c20 2024-04-23T18:36:01.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.221+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2275837438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f081a00a0 con 0x7f4f08103c20 2024-04-23T18:36:01.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.221+0000 7f4efe7fc640 1 -- 172.21.15.132:0/2275837438 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f00005020 con 0x7f4f08103c20 2024-04-23T18:36:01.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.221+0000 7f4efe7fc640 1 -- 172.21.15.132:0/2275837438 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00010690 con 0x7f4f08103c20 2024-04-23T18:36:01.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.221+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2275837438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f081a0640 con 0x7f4f08103c20 2024-04-23T18:36:01.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.222+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2275837438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f08109390 con 0x7f4f08103c20 2024-04-23T18:36:01.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.223+0000 7f4efe7fc640 1 -- 172.21.15.132:0/2275837438 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f4f000054d0 con 0x7f4f08103c20 2024-04-23T18:36:01.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.224+0000 7f4efe7fc640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4ef40a3ca0 0x7f4ef40a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.224+0000 7f4f0d784640 1 -- 172.21.15.132:0/2275837438 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4ef40a3ca0 msgr2=0x7f4ef40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/636352941 2024-04-23T18:36:01.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.224+0000 7f4efe7fc640 1 -- 172.21.15.132:0/2275837438 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4f000d0b50 con 0x7f4f08103c20 2024-04-23T18:36:01.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.224+0000 7f4f0d784640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4ef40a3ca0 0x7f4ef40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:36:01.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.229+0000 7f4efe7fc640 1 -- 172.21.15.132:0/2275837438 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f0009d4e0 con 0x7f4f08103c20 2024-04-23T18:36:01.254 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:36:00 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:36:00.913+0000 7fe7957da200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:36:01.254 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:36:01 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:36:01.047+0000 7fe7957da200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:36:01.254 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:36:01 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:36:01.142+0000 7fe7957da200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:36:01.254 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:36:01 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:36:01.261+0000 7fe7957da200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:36:01.263 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:36:00 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:36:00.925+0000 7f132814a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:36:01.263 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:36:01 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:36:01.059+0000 7f132814a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:36:01.264 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:36:01 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:36:01.154+0000 7f132814a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:36:01.264 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:36:01 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:36:01.271+0000 7f132814a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:36:01.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.425+0000 7f4f0d784640 1 -- 172.21.15.132:0/2275837438 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4ef40a3ca0 msgr2=0x7f4ef40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/636352941 2024-04-23T18:36:01.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.425+0000 7f4f0d784640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4ef40a3ca0 0x7f4ef40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:36:01.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.453+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2275837438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f4f08062b50 con 0x7f4f08103c20 2024-04-23T18:36:01.498 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:01 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:01.192+0000 7fa7b6a36200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:36:01.498 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:01 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:01.337+0000 7fa7b6a36200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:36:01.498 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:01 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:01.437+0000 7fa7b6a36200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:36:01.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.515+0000 7f4efe7fc640 1 -- 172.21.15.132:0/2275837438 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f4f0009cde0 con 0x7f4f08103c20 2024-04-23T18:36:01.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.517+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2275837438 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4ef40a3ca0 msgr2=0x7f4ef40a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:36:01.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.517+0000 7f4f0f9e8640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4ef40a3ca0 0x7f4ef40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.517+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2275837438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f08103c20 msgr2=0x7f4f0819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.517+0000 7f4f0f9e8640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f08103c20 0x7f4f0819f740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009fd0 tx=0x7f4f0000ef10 comp rx=0 tx=0).stop 2024-04-23T18:36:01.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.517+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2275837438 shutdown_connections 2024-04-23T18:36:01.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.517+0000 7f4f0f9e8640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7f4ef40a3ca0 0x7f4ef40a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.517+0000 7f4f0f9e8640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f08103c20 0x7f4f0819f740 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.517+0000 7f4f0f9e8640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f08103260 0x7f4f081986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.517+0000 7f4f0f9e8640 1 --2- 172.21.15.132:0/2275837438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f08100290 0x7f4f08198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.517+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2275837438 >> 172.21.15.132:0/2275837438 conn(0x7f4f080fbbc0 msgr2=0x7f4f08106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:01.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.518+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2275837438 shutdown_connections 2024-04-23T18:36:01.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.518+0000 7f4f0f9e8640 1 -- 172.21.15.132:0/2275837438 wait complete. 2024-04-23T18:36:01.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:01 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2275837438' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:36:01.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:01 smithi148 ceph-mon[32311]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:36:01.520 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-23T18:36:01.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:01 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2275837438' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:36:01.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:01 smithi157 ceph-mon[32580]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:36:01.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.632+0000 7fa71a82f640 1 -- 172.21.15.132:0/710153184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7141031d0 msgr2=0x7fa714103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.632+0000 7fa71a82f640 1 --2- 172.21.15.132:0/710153184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7141031d0 0x7fa714103650 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa708009900 tx=0x7fa70802a410 comp rx=0 tx=0).stop 2024-04-23T18:36:01.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.632+0000 7fa71a82f640 1 -- 172.21.15.132:0/710153184 shutdown_connections 2024-04-23T18:36:01.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.632+0000 7fa71a82f640 1 --2- 172.21.15.132:0/710153184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa714103b90 0x7fa71410a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.632+0000 7fa71a82f640 1 --2- 172.21.15.132:0/710153184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7141031d0 0x7fa714103650 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.632+0000 7fa71a82f640 1 --2- 172.21.15.132:0/710153184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa714101fe0 0x7fa714102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.632+0000 7fa71a82f640 1 -- 172.21.15.132:0/710153184 >> 172.21.15.132:0/710153184 conn(0x7fa7140fd570 msgr2=0x7fa7140ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:01.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.632+0000 7fa71a82f640 1 -- 172.21.15.132:0/710153184 shutdown_connections 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.632+0000 7fa71a82f640 1 -- 172.21.15.132:0/710153184 wait complete. 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.632+0000 7fa71a82f640 1 Processor -- start 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.632+0000 7fa71a82f640 1 -- start start 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa71a82f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa714101fe0 0x7fa7141980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa71a82f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7141031d0 0x7fa714198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa71a82f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa714103b90 0x7fa71419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa71a82f640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa71410db20 con 0x7fa7141031d0 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa71a82f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa71410d9a0 con 0x7fa714101fe0 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa71a82f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa71410dca0 con 0x7fa714103b90 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa7137fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7141031d0 0x7fa714198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa7137fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7141031d0 0x7fa714198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57458/0 (socket says 172.21.15.132:57458) 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa7137fe640 1 -- 172.21.15.132:0/1917569916 learned_addr learned my addr 172.21.15.132:0/1917569916 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa713fff640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa714101fe0 0x7fa7141980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa718dcc640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa714103b90 0x7fa71419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa7137fe640 1 -- 172.21.15.132:0/1917569916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa714103b90 msgr2=0x7fa71419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa7137fe640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa714103b90 0x7fa71419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa7137fe640 1 -- 172.21.15.132:0/1917569916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa714101fe0 msgr2=0x7fa7141980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa7137fe640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa714101fe0 0x7fa7141980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa7137fe640 1 -- 172.21.15.132:0/1917569916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71419fd10 con 0x7fa7141031d0 2024-04-23T18:36:01.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa7137fe640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7141031d0 0x7fa714198630 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa708005940 tx=0x7fa708004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:01.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.633+0000 7fa7117fa640 1 -- 172.21.15.132:0/1917569916 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70803f030 con 0x7fa7141031d0 2024-04-23T18:36:01.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.634+0000 7fa71a82f640 1 -- 172.21.15.132:0/1917569916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71419ffa0 con 0x7fa7141031d0 2024-04-23T18:36:01.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.634+0000 7fa7117fa640 1 -- 172.21.15.132:0/1917569916 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa708003cb0 con 0x7fa7141031d0 2024-04-23T18:36:01.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.634+0000 7fa7117fa640 1 -- 172.21.15.132:0/1917569916 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708035070 con 0x7fa7141031d0 2024-04-23T18:36:01.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.634+0000 7fa71a82f640 1 -- 172.21.15.132:0/1917569916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7141a0400 con 0x7fa7141031d0 2024-04-23T18:36:01.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.635+0000 7fa71a82f640 1 -- 172.21.15.132:0/1917569916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa714109300 con 0x7fa7141031d0 2024-04-23T18:36:01.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.636+0000 7fa7117fa640 1 -- 172.21.15.132:0/1917569916 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7fa708042030 con 0x7fa7141031d0 2024-04-23T18:36:01.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.636+0000 7fa7117fa640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6fc0a3ca0 0x7fa6fc0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.636+0000 7fa713fff640 1 -- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6fc0a3ca0 msgr2=0x7fa6fc0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/636352941 2024-04-23T18:36:01.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.636+0000 7fa7117fa640 1 -- 172.21.15.132:0/1917569916 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa7080f0cc0 con 0x7fa7141031d0 2024-04-23T18:36:01.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.636+0000 7fa713fff640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6fc0a3ca0 0x7fa6fc0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:36:01.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.641+0000 7fa7117fa640 1 -- 172.21.15.132:0/1917569916 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa708003e60 con 0x7fa7141031d0 2024-04-23T18:36:01.829 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.837+0000 7fa713fff640 1 -- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6fc0a3ca0 msgr2=0x7fa6fc0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/636352941 2024-04-23T18:36:01.829 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.837+0000 7fa713fff640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6fc0a3ca0 0x7fa6fc0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:36:01.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:01.851+0000 7fa71a82f640 1 -- 172.21.15.132:0/1917569916 --> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fa7140624f0 con 0x7fa6fc0a3ca0 2024-04-23T18:36:01.861 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:01 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:01.561+0000 7fa7b6a36200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:36:01.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:01 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2275837438' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:36:01.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:01 smithi132 ceph-mon[29178]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:36:02.014 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:36:01 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:36:01.579+0000 7f132814a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:36:02.039 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:36:01 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:36:01.568+0000 7fe7957da200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:36:02.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:02.238+0000 7fa713fff640 1 -- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6fc0a3ca0 msgr2=0x7fa6fc0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/636352941 2024-04-23T18:36:02.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:02.238+0000 7fa713fff640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6fc0a3ca0 0x7fa6fc0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T18:36:02.361 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:01 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:01.873+0000 7fa7b6a36200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:36:02.506 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:36:02.173+0000 7fe7957da200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:36:02.506 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z[34289]: 2024-04-23T18:36:02.388+0000 7fe7957da200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:36:02.507 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:36:02.181+0000 7f132814a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:36:02.507 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y[33691]: 2024-04-23T18:36:02.394+0000 7f132814a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:36:02.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:02.528+0000 7fa7117fa640 1 -- 172.21.15.132:0/1917569916 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 20) ==== 138664+0+0 (secure 0 0 0) 0x7fa70803bd00 con 0x7fa7141031d0 2024-04-23T18:36:02.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:02.728+0000 7fa7117fa640 1 -- 172.21.15.132:0/1917569916 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fa708003800 con 0x7fa7141031d0 2024-04-23T18:36:02.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:02.728+0000 7fa7117fa640 1 -- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6fc0a3ca0 msgr2=0x7fa6fc0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:36:02.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:02.728+0000 7fa7117fa640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/636352941,v1:172.21.15.132:6801/636352941] conn(0x7fa6fc0a3ca0 0x7fa6fc0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:02.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T18:36:02.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:36:02.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: Standby manager daemon z restarted 2024-04-23T18:36:02.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: Standby manager daemon z started 2024-04-23T18:36:02.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.157:0/1499795231' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:36:02.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.157:0/1499795231' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:36:02.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.157:0/1499795231' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:36:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.157:0/1499795231' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:36:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: Standby manager daemon y restarted 2024-04-23T18:36:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: Standby manager daemon y started 2024-04-23T18:36:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.148:0/239165865' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:36:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.148:0/239165865' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:36:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.148:0/239165865' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:36:02.765 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:02 smithi148 ceph-mon[32311]: from='mgr.? 172.21.15.148:0/239165865' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:36:02.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T18:36:02.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:36:02.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: Standby manager daemon z restarted 2024-04-23T18:36:02.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: Standby manager daemon z started 2024-04-23T18:36:02.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.157:0/1499795231' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:36:02.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.157:0/1499795231' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:36:02.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.157:0/1499795231' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:36:02.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.157:0/1499795231' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:36:02.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: Standby manager daemon y restarted 2024-04-23T18:36:02.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: Standby manager daemon y started 2024-04-23T18:36:02.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.148:0/239165865' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:36:02.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.148:0/239165865' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:36:02.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.148:0/239165865' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:36:02.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:02 smithi157 ceph-mon[32580]: from='mgr.? 172.21.15.148:0/239165865' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:36:02.859 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:02.487+0000 7fa7b6a36200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:36:02.860 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:02.714+0000 7fa7b6a36200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:36:02.860 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T18:36:02.860 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:36:02.860 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: Standby manager daemon z restarted 2024-04-23T18:36:02.860 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: Standby manager daemon z started 2024-04-23T18:36:02.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.157:0/1499795231' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:36:02.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.157:0/1499795231' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:36:02.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.157:0/1499795231' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:36:02.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.157:0/1499795231' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:36:02.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: Standby manager daemon y restarted 2024-04-23T18:36:02.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: Standby manager daemon y started 2024-04-23T18:36:02.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.148:0/239165865' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:36:02.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.148:0/239165865' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:36:02.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.148:0/239165865' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:36:02.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-mon[29178]: from='mgr.? 172.21.15.148:0/239165865' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:36:03.111 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:02.952+0000 7fa715bd3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.111 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:02.952+0000 7fa715bd3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.111 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:02.952+0000 7fa715bd3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.111 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:02.952+0000 7fa715bd3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:02.952+0000 7fa715bd3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:02.953+0000 7fa7193da700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:02.953+0000 7fa7193da700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:02.953+0000 7fa7193da700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:02.953+0000 7fa7193da700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:02.953+0000 7fa7193da700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.737+0000 7fa7117fa640 1 -- 172.21.15.132:0/1917569916 <== mon.0 v2:172.21.15.132:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fa708097b30 con 0x7fa7141031d0 2024-04-23T18:36:03.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.738+0000 7fa7117fa640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa6fc034f60 0x7fa6fc035340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:03.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.738+0000 7fa7117fa640 1 -- 172.21.15.132:0/1917569916 --> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fa7140624f0 con 0x7fa6fc034f60 2024-04-23T18:36:03.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.739+0000 7fa713fff640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa6fc034f60 0x7fa6fc035340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:03.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.739+0000 7fa713fff640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa6fc034f60 0x7fa6fc035340 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa704005fd0 tx=0x7fa70400b410 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:03.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.762+0000 7fa7117fa640 1 -- 172.21.15.132:0/1917569916 <== mgr.15366 v2:172.21.15.132:6800/3428544997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fa7140624f0 con 0x7fa6fc034f60 2024-04-23T18:36:03.754 INFO:teuthology.orchestra.run.smithi132.stdout:{} 2024-04-23T18:36:03.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.765+0000 7fa71a82f640 1 -- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa6fc034f60 msgr2=0x7fa6fc035340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:03.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.765+0000 7fa71a82f640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa6fc034f60 0x7fa6fc035340 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa704005fd0 tx=0x7fa70400b410 comp rx=0 tx=0).stop 2024-04-23T18:36:03.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.765+0000 7fa71a82f640 1 -- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7141031d0 msgr2=0x7fa714198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:03.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.765+0000 7fa71a82f640 1 --2- 172.21.15.132:0/1917569916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7141031d0 0x7fa714198630 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa708005940 tx=0x7fa708004430 comp rx=0 tx=0).stop 2024-04-23T18:36:03.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.765+0000 7fa718dcc640 1 -- 172.21.15.132:0/1917569916 reap_dead start 2024-04-23T18:36:03.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.765+0000 7fa71a82f640 1 -- 172.21.15.132:0/1917569916 shutdown_connections 2024-04-23T18:36:03.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.765+0000 7fa71a82f640 1 -- 172.21.15.132:0/1917569916 >> 172.21.15.132:0/1917569916 conn(0x7fa7140fd570 msgr2=0x7fa7141064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:03.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.765+0000 7fa71a82f640 1 -- 172.21.15.132:0/1917569916 shutdown_connections 2024-04-23T18:36:03.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.765+0000 7fa71a82f640 1 -- 172.21.15.132:0/1917569916 wait complete. 2024-04-23T18:36:03.768 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:03.746+0000 7fa7133ce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.769 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:03.746+0000 7fa7133ce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.769 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:03.746+0000 7fa7133ce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.769 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:03.746+0000 7fa7133ce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.769 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:36:03.746+0000 7fa7133ce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:36:03.769 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-23T18:36:03.769 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: Active manager daemon x restarted 2024-04-23T18:36:03.769 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: Activating manager daemon x 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: mgrmap e21: x(active, starting, since 0.0121493s), standbys: y, z 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:36:03.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:36:03.771 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:36:03.772 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:36:03.772 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:36:03.772 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:36:03.772 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:36:03.772 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:36:03.772 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:36:03.772 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:36:03.772 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: Manager daemon x is now available 2024-04-23T18:36:03.772 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:36:03.772 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:36:03.772 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:03 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:36:03.773 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-23T18:36:03.783 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T18:36:03.796 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:36:03.796 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-04-23T18:36:03.796 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-23T18:36:03.797 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-23T18:36:03.797 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: Active manager daemon x restarted 2024-04-23T18:36:03.797 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: Activating manager daemon x 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: mgrmap e21: x(active, starting, since 0.0121493s), standbys: y, z 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:36:03.798 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:36:03.799 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:36:03.800 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:36:03.800 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:36:03.800 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:36:03.800 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:36:03.800 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:36:03.800 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:36:03.800 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: Manager daemon x is now available 2024-04-23T18:36:03.800 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:36:03.800 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:36:03.800 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:03 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:36:03.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.909+0000 7fa79a115640 1 -- 172.21.15.132:0/3756390614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa794100640 msgr2=0x7fa794100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:03.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.909+0000 7fa79a115640 1 --2- 172.21.15.132:0/3756390614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa794100640 0x7fa794100a60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa78400aa10 tx=0x7fa78402b720 comp rx=0 tx=0).stop 2024-04-23T18:36:03.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 -- 172.21.15.132:0/3756390614 shutdown_connections 2024-04-23T18:36:03.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 --2- 172.21.15.132:0/3756390614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa794101830 0x7fa794101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 --2- 172.21.15.132:0/3756390614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7941021f0 0x7fa794108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 --2- 172.21.15.132:0/3756390614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa794100640 0x7fa794100a60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 -- 172.21.15.132:0/3756390614 >> 172.21.15.132:0/3756390614 conn(0x7fa7940fbbb0 msgr2=0x7fa7940fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:03.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 -- 172.21.15.132:0/3756390614 shutdown_connections 2024-04-23T18:36:03.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 -- 172.21.15.132:0/3756390614 wait complete. 2024-04-23T18:36:03.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 Processor -- start 2024-04-23T18:36:03.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 -- start start 2024-04-23T18:36:03.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa794100640 0x7fa7941981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:03.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa794101830 0x7fa794198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7941021f0 0x7fa79419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa79410b900 con 0x7fa7941021f0 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa79410b780 con 0x7fa794101830 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.910+0000 7fa79a115640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa79410ba80 con 0x7fa794100640 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa793fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7941021f0 0x7fa79419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa793fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7941021f0 0x7fa79419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57620/0 (socket says 172.21.15.132:57620) 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa793fff640 1 -- 172.21.15.132:0/3968096370 learned_addr learned my addr 172.21.15.132:0/3968096370 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa7937fe640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa794100640 0x7fa7941981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa78affd640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa794101830 0x7fa794198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa793fff640 1 -- 172.21.15.132:0/3968096370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa794100640 msgr2=0x7fa7941981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa793fff640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa794100640 0x7fa7941981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa793fff640 1 -- 172.21.15.132:0/3968096370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa794101830 msgr2=0x7fa794198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa793fff640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa794101830 0x7fa794198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa793fff640 1 -- 172.21.15.132:0/3968096370 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79419fd90 con 0x7fa7941021f0 2024-04-23T18:36:03.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa78affd640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa794101830 0x7fa794198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:03.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa793fff640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7941021f0 0x7fa79419f720 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00cad0 tx=0x7fa78c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:03.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa7917fa640 1 -- 172.21.15.132:0/3968096370 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00ae20 con 0x7fa7941021f0 2024-04-23T18:36:03.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa7917fa640 1 -- 172.21.15.132:0/3968096370 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa78c005020 con 0x7fa7941021f0 2024-04-23T18:36:03.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa7917fa640 1 -- 172.21.15.132:0/3968096370 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00f6b0 con 0x7fa7941021f0 2024-04-23T18:36:03.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa79a115640 1 -- 172.21.15.132:0/3968096370 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7941a0080 con 0x7fa7941021f0 2024-04-23T18:36:03.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.911+0000 7fa79a115640 1 -- 172.21.15.132:0/3968096370 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7941a0580 con 0x7fa7941021f0 2024-04-23T18:36:03.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.913+0000 7fa79a115640 1 -- 172.21.15.132:0/3968096370 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa794107a00 con 0x7fa7941021f0 2024-04-23T18:36:03.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.913+0000 7fa7917fa640 1 -- 172.21.15.132:0/3968096370 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fa78c0051d0 con 0x7fa7941021f0 2024-04-23T18:36:03.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.914+0000 7fa7917fa640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa7800a3a60 0x7fa7800a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:03.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.914+0000 7fa7937fe640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa7800a3a60 0x7fa7800a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:03.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.914+0000 7fa7917fa640 1 -- 172.21.15.132:0/3968096370 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fa78c0d02a0 con 0x7fa7941021f0 2024-04-23T18:36:03.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.915+0000 7fa7937fe640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa7800a3a60 0x7fa7800a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa784000c00 tx=0x7fa7840305b0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:03.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:03.919+0000 7fa7917fa640 1 -- 172.21.15.132:0/3968096370 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa78c014030 con 0x7fa7941021f0 2024-04-23T18:36:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-23T18:36:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: Active manager daemon x restarted 2024-04-23T18:36:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: Activating manager daemon x 2024-04-23T18:36:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T18:36:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: mgrmap e21: x(active, starting, since 0.0121493s), standbys: y, z 2024-04-23T18:36:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:36:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:36:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:36:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:36:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:36:03.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:36:03.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:36:03.958 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:36:03.958 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:36:03.958 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: Manager daemon x is now available 2024-04-23T18:36:03.958 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:36:03.958 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:36:03.958 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:03 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:36:04.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.130+0000 7fa79a115640 1 -- 172.21.15.132:0/3968096370 --> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fa794062390 con 0x7fa7800a3a60 2024-04-23T18:36:04.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.192+0000 7fa7917fa640 1 -- 172.21.15.132:0/3968096370 <== mgr.15366 v2:172.21.15.132:6800/3428544997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa794062390 con 0x7fa7800a3a60 2024-04-23T18:36:04.184 INFO:teuthology.orchestra.run.smithi132.stdout:Schedule set for path / 2024-04-23T18:36:04.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.194+0000 7fa79a115640 1 -- 172.21.15.132:0/3968096370 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa7800a3a60 msgr2=0x7fa7800a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:04.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.194+0000 7fa79a115640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa7800a3a60 0x7fa7800a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa784000c00 tx=0x7fa7840305b0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.194+0000 7fa79a115640 1 -- 172.21.15.132:0/3968096370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7941021f0 msgr2=0x7fa79419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:04.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.194+0000 7fa79a115640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7941021f0 0x7fa79419f720 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00cad0 tx=0x7fa78c00cf90 comp rx=0 tx=0).stop 2024-04-23T18:36:04.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.194+0000 7fa79a115640 1 -- 172.21.15.132:0/3968096370 shutdown_connections 2024-04-23T18:36:04.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.194+0000 7fa79a115640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa7800a3a60 0x7fa7800a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.194+0000 7fa79a115640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7941021f0 0x7fa79419f720 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.194+0000 7fa79a115640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa794101830 0x7fa794198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.194+0000 7fa79a115640 1 --2- 172.21.15.132:0/3968096370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa794100640 0x7fa7941981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.194+0000 7fa79a115640 1 -- 172.21.15.132:0/3968096370 >> 172.21.15.132:0/3968096370 conn(0x7fa7940fbbb0 msgr2=0x7fa794104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:04.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.194+0000 7fa79a115640 1 -- 172.21.15.132:0/3968096370 shutdown_connections 2024-04-23T18:36:04.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.194+0000 7fa79a115640 1 -- 172.21.15.132:0/3968096370 wait complete. 2024-04-23T18:36:04.197 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-23T18:36:04.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.307+0000 7fe113c2c640 1 -- 172.21.15.132:0/3815088209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c100290 msgr2=0x7fe10c1006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:04.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.307+0000 7fe113c2c640 1 --2- 172.21.15.132:0/3815088209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c100290 0x7fe10c1006b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe108009900 tx=0x7fe10802a410 comp rx=0 tx=0).stop 2024-04-23T18:36:04.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.307+0000 7fe113c2c640 1 -- 172.21.15.132:0/3815088209 shutdown_connections 2024-04-23T18:36:04.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.307+0000 7fe113c2c640 1 --2- 172.21.15.132:0/3815088209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10c1019b0 0x7fe10c106250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.307+0000 7fe113c2c640 1 --2- 172.21.15.132:0/3815088209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe10c100ff0 0x7fe10c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.307+0000 7fe113c2c640 1 --2- 172.21.15.132:0/3815088209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c100290 0x7fe10c1006b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.307+0000 7fe113c2c640 1 -- 172.21.15.132:0/3815088209 >> 172.21.15.132:0/3815088209 conn(0x7fe10c0fbbc0 msgr2=0x7fe10c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:04.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.307+0000 7fe113c2c640 1 -- 172.21.15.132:0/3815088209 shutdown_connections 2024-04-23T18:36:04.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.307+0000 7fe113c2c640 1 -- 172.21.15.132:0/3815088209 wait complete. 2024-04-23T18:36:04.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.307+0000 7fe113c2c640 1 Processor -- start 2024-04-23T18:36:04.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe113c2c640 1 -- start start 2024-04-23T18:36:04.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe113c2c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c100290 0x7fe10c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:04.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe113c2c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10c100ff0 0x7fe10c19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:04.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe113c2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe10c1019b0 0x7fe10c1a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe113c2c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c114110 con 0x7fe10c100290 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe113c2c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c113f90 con 0x7fe10c1019b0 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe113c2c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c114290 con 0x7fe10c100ff0 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe1119c8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c100290 0x7fe10c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe1119c8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c100290 0x7fe10c19e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57634/0 (socket says 172.21.15.132:57634) 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe1119c8640 1 -- 172.21.15.132:0/64075399 learned_addr learned my addr 172.21.15.132:0/64075399 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe1111c7640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10c100ff0 0x7fe10c19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe1121c9640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe10c1019b0 0x7fe10c1a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe1119c8640 1 -- 172.21.15.132:0/64075399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10c100ff0 msgr2=0x7fe10c19ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe1119c8640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10c100ff0 0x7fe10c19ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe1119c8640 1 -- 172.21.15.132:0/64075399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe10c1019b0 msgr2=0x7fe10c1a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe1119c8640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe10c1019b0 0x7fe10c1a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe1119c8640 1 -- 172.21.15.132:0/64075399 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10c1a6320 con 0x7fe10c100290 2024-04-23T18:36:04.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe1119c8640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c100290 0x7fe10c19e700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe108005940 tx=0x7fe1080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:04.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe102ffd640 1 -- 172.21.15.132:0/64075399 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10803f030 con 0x7fe10c100290 2024-04-23T18:36:04.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe113c2c640 1 -- 172.21.15.132:0/64075399 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10c1a65b0 con 0x7fe10c100290 2024-04-23T18:36:04.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe102ffd640 1 -- 172.21.15.132:0/64075399 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe108003cb0 con 0x7fe10c100290 2024-04-23T18:36:04.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe102ffd640 1 -- 172.21.15.132:0/64075399 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108035070 con 0x7fe10c100290 2024-04-23T18:36:04.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.308+0000 7fe113c2c640 1 -- 172.21.15.132:0/64075399 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10c1a6af0 con 0x7fe10c100290 2024-04-23T18:36:04.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.309+0000 7fe113c2c640 1 -- 172.21.15.132:0/64075399 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10c105060 con 0x7fe10c100290 2024-04-23T18:36:04.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.311+0000 7fe102ffd640 1 -- 172.21.15.132:0/64075399 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fe108042030 con 0x7fe10c100290 2024-04-23T18:36:04.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.311+0000 7fe102ffd640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe0f80a3a60 0x7fe0f80a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:04.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.312+0000 7fe1111c7640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe0f80a3a60 0x7fe0f80a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:04.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.312+0000 7fe102ffd640 1 -- 172.21.15.132:0/64075399 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fe1080f0b50 con 0x7fe10c100290 2024-04-23T18:36:04.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.312+0000 7fe1111c7640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe0f80a3a60 0x7fe0f80a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe10c19fc10 tx=0x7fe0fc005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:04.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.315+0000 7fe102ffd640 1 -- 172.21.15.132:0/64075399 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe108003e60 con 0x7fe10c100290 2024-04-23T18:36:04.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.554+0000 7fe113c2c640 1 -- 172.21.15.132:0/64075399 --> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fe10c0624f0 con 0x7fe0f80a3a60 2024-04-23T18:36:04.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.614+0000 7fe102ffd640 1 -- 172.21.15.132:0/64075399 <== mgr.15366 v2:172.21.15.132:6800/3428544997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fe10c0624f0 con 0x7fe0f80a3a60 2024-04-23T18:36:04.606 INFO:teuthology.orchestra.run.smithi132.stdout:Retention added to path / 2024-04-23T18:36:04.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.616+0000 7fe113c2c640 1 -- 172.21.15.132:0/64075399 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe0f80a3a60 msgr2=0x7fe0f80a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:04.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.616+0000 7fe113c2c640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe0f80a3a60 0x7fe0f80a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe10c19fc10 tx=0x7fe0fc005f90 comp rx=0 tx=0).stop 2024-04-23T18:36:04.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.616+0000 7fe113c2c640 1 -- 172.21.15.132:0/64075399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c100290 msgr2=0x7fe10c19e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:04.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.616+0000 7fe113c2c640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c100290 0x7fe10c19e700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe108005940 tx=0x7fe1080042f0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.616+0000 7fe113c2c640 1 -- 172.21.15.132:0/64075399 shutdown_connections 2024-04-23T18:36:04.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.616+0000 7fe113c2c640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe0f80a3a60 0x7fe0f80a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.616+0000 7fe113c2c640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10c100ff0 0x7fe10c19ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.616+0000 7fe113c2c640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe10c1019b0 0x7fe10c1a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.616+0000 7fe113c2c640 1 --2- 172.21.15.132:0/64075399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c100290 0x7fe10c19e700 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.616+0000 7fe113c2c640 1 -- 172.21.15.132:0/64075399 >> 172.21.15.132:0/64075399 conn(0x7fe10c0fbbc0 msgr2=0x7fe10c102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:04.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.616+0000 7fe113c2c640 1 -- 172.21.15.132:0/64075399 shutdown_connections 2024-04-23T18:36:04.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.616+0000 7fe113c2c640 1 -- 172.21.15.132:0/64075399 wait complete. 2024-04-23T18:36:04.620 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-23T18:36:04.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2530737556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d30102050 msgr2=0x7f7d301024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:04.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 --2- 172.21.15.132:0/2530737556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d30102050 0x7f7d301024d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009900 tx=0x7f7d1c02a410 comp rx=0 tx=0).stop 2024-04-23T18:36:04.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2530737556 shutdown_connections 2024-04-23T18:36:04.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 --2- 172.21.15.132:0/2530737556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d30102a10 0x7f7d30109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 --2- 172.21.15.132:0/2530737556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d30102050 0x7f7d301024d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 --2- 172.21.15.132:0/2530737556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d30100e70 0x7f7d30101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2530737556 >> 172.21.15.132:0/2530737556 conn(0x7f7d300fc430 msgr2=0x7f7d300fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:04.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2530737556 shutdown_connections 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2530737556 wait complete. 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 Processor -- start 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 -- start start 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d30100e70 0x7f7d30197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d30102050 0x7f7d30197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d30102a10 0x7f7d3019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3010ca10 con 0x7f7d30100e70 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3010c890 con 0x7f7d30102050 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.737+0000 7f7d36d6f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3010cb90 con 0x7f7d30102a10 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d34b0b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d30100e70 0x7f7d30197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d34b0b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d30100e70 0x7f7d30197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57656/0 (socket says 172.21.15.132:57656) 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d34b0b640 1 -- 172.21.15.132:0/2369133368 learned_addr learned my addr 172.21.15.132:0/2369133368 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d3530c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d30102a10 0x7f7d3019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:04.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d2ffff640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d30102050 0x7f7d30197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:04.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d34b0b640 1 -- 172.21.15.132:0/2369133368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d30102a10 msgr2=0x7f7d3019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:04.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d34b0b640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d30102a10 0x7f7d3019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d34b0b640 1 -- 172.21.15.132:0/2369133368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d30102050 msgr2=0x7f7d30197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:04.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d34b0b640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d30102050 0x7f7d30197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d34b0b640 1 -- 172.21.15.132:0/2369133368 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3019ec20 con 0x7f7d30100e70 2024-04-23T18:36:04.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d3530c640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d30102a10 0x7f7d3019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:04.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d34b0b640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d30100e70 0x7f7d30197050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400ead0 tx=0x7f7d2400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:04.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d2dffb640 1 -- 172.21.15.132:0/2369133368 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400bbd0 con 0x7f7d30100e70 2024-04-23T18:36:04.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2369133368 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3006a230 con 0x7f7d30100e70 2024-04-23T18:36:04.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d2dffb640 1 -- 172.21.15.132:0/2369133368 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2400bd70 con 0x7f7d30100e70 2024-04-23T18:36:04.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.738+0000 7f7d2dffb640 1 -- 172.21.15.132:0/2369133368 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24017410 con 0x7f7d30100e70 2024-04-23T18:36:04.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.739+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2369133368 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3006a7a0 con 0x7f7d30100e70 2024-04-23T18:36:04.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.739+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2369133368 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3004fce0 con 0x7f7d30100e70 2024-04-23T18:36:04.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.741+0000 7f7d2dffb640 1 -- 172.21.15.132:0/2369133368 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f7d2401e020 con 0x7f7d30100e70 2024-04-23T18:36:04.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.741+0000 7f7d2dffb640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f7d200a3a60 0x7f7d200a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:04.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.741+0000 7f7d2ffff640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f7d200a3a60 0x7f7d200a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:04.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.741+0000 7f7d2dffb640 1 -- 172.21.15.132:0/2369133368 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f7d240d1780 con 0x7f7d30100e70 2024-04-23T18:36:04.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.742+0000 7f7d2ffff640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f7d200a3a60 0x7f7d200a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c000c00 tx=0x7f7d1c0314a0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:04.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.746+0000 7f7d2dffb640 1 -- 172.21.15.132:0/2369133368 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d2409e320 con 0x7f7d30100e70 2024-04-23T18:36:04.780 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:04 smithi157 ceph-mon[32580]: [23/Apr/2024:18:36:03] ENGINE Bus STARTING 2024-04-23T18:36:04.780 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:04 smithi157 ceph-mon[32580]: [23/Apr/2024:18:36:03] ENGINE Serving on http://172.21.15.132:8765 2024-04-23T18:36:04.780 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:04 smithi157 ceph-mon[32580]: [23/Apr/2024:18:36:03] ENGINE Serving on https://172.21.15.132:7150 2024-04-23T18:36:04.780 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:04 smithi157 ceph-mon[32580]: [23/Apr/2024:18:36:03] ENGINE Bus STARTED 2024-04-23T18:36:04.781 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:04 smithi157 ceph-mon[32580]: mgrmap e22: x(active, since 1.01816s), standbys: y, z 2024-04-23T18:36:04.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:04 smithi132 ceph-mon[29178]: [23/Apr/2024:18:36:03] ENGINE Bus STARTING 2024-04-23T18:36:04.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:04 smithi132 ceph-mon[29178]: [23/Apr/2024:18:36:03] ENGINE Serving on http://172.21.15.132:8765 2024-04-23T18:36:04.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:04 smithi132 ceph-mon[29178]: [23/Apr/2024:18:36:03] ENGINE Serving on https://172.21.15.132:7150 2024-04-23T18:36:04.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:04 smithi132 ceph-mon[29178]: [23/Apr/2024:18:36:03] ENGINE Bus STARTED 2024-04-23T18:36:04.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:04 smithi132 ceph-mon[29178]: mgrmap e22: x(active, since 1.01816s), standbys: y, z 2024-04-23T18:36:04.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.975+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2369133368 --> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7d3006aa00 con 0x7f7d200a3a60 2024-04-23T18:36:04.974 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.982+0000 7f7d2dffb640 1 -- 172.21.15.132:0/2369133368 <== mgr.15366 v2:172.21.15.132:6800/3428544997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f7d3006aa00 con 0x7f7d200a3a60 2024-04-23T18:36:04.975 INFO:teuthology.orchestra.run.smithi132.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-23T00:00:00", "created": "2024-04-23T18:36:04", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-23T18:36:04.977 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.985+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2369133368 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f7d200a3a60 msgr2=0x7f7d200a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:04.977 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.985+0000 7f7d36d6f640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f7d200a3a60 0x7f7d200a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c000c00 tx=0x7f7d1c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.977 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.985+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2369133368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d30100e70 msgr2=0x7f7d30197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:04.977 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.985+0000 7f7d36d6f640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d30100e70 0x7f7d30197050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400ead0 tx=0x7f7d2400ef90 comp rx=0 tx=0).stop 2024-04-23T18:36:04.977 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.985+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2369133368 shutdown_connections 2024-04-23T18:36:04.977 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.985+0000 7f7d36d6f640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f7d200a3a60 0x7f7d200a5f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.985+0000 7f7d36d6f640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d30102a10 0x7f7d3019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.985+0000 7f7d36d6f640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d30102050 0x7f7d30197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.985+0000 7f7d36d6f640 1 --2- 172.21.15.132:0/2369133368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d30100e70 0x7f7d30197050 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:04.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.985+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2369133368 >> 172.21.15.132:0/2369133368 conn(0x7f7d300fc430 msgr2=0x7f7d301053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:04.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.985+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2369133368 shutdown_connections 2024-04-23T18:36:04.978 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:04.985+0000 7f7d36d6f640 1 -- 172.21.15.132:0/2369133368 wait complete. 2024-04-23T18:36:04.988 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-23T18:36:05.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:04 smithi148 ceph-mon[32311]: [23/Apr/2024:18:36:03] ENGINE Bus STARTING 2024-04-23T18:36:05.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:04 smithi148 ceph-mon[32311]: [23/Apr/2024:18:36:03] ENGINE Serving on http://172.21.15.132:8765 2024-04-23T18:36:05.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:04 smithi148 ceph-mon[32311]: [23/Apr/2024:18:36:03] ENGINE Serving on https://172.21.15.132:7150 2024-04-23T18:36:05.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:04 smithi148 ceph-mon[32311]: [23/Apr/2024:18:36:03] ENGINE Bus STARTED 2024-04-23T18:36:05.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:04 smithi148 ceph-mon[32311]: mgrmap e22: x(active, since 1.01816s), standbys: y, z 2024-04-23T18:36:05.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.098+0000 7f594816f640 1 -- 172.21.15.132:0/1080357188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59401032f0 msgr2=0x7f5940103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.098+0000 7f594816f640 1 --2- 172.21.15.132:0/1080357188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59401032f0 0x7f5940103770 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5930009930 tx=0x7f593002a420 comp rx=0 tx=0).stop 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.098+0000 7f594816f640 1 -- 172.21.15.132:0/1080357188 shutdown_connections 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.098+0000 7f594816f640 1 --2- 172.21.15.132:0/1080357188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5940103cb0 0x7f594010a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.098+0000 7f594816f640 1 --2- 172.21.15.132:0/1080357188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59401032f0 0x7f5940103770 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.098+0000 7f594816f640 1 --2- 172.21.15.132:0/1080357188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5940100280 0x7f59401006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.098+0000 7f594816f640 1 -- 172.21.15.132:0/1080357188 >> 172.21.15.132:0/1080357188 conn(0x7f59400fbbb0 msgr2=0x7f59400fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.098+0000 7f594816f640 1 -- 172.21.15.132:0/1080357188 shutdown_connections 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.098+0000 7f594816f640 1 -- 172.21.15.132:0/1080357188 wait complete. 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594816f640 1 Processor -- start 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594816f640 1 -- start start 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594816f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5940100280 0x7f59401981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594816f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59401032f0 0x7f5940198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594816f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5940103cb0 0x7f594019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594816f640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f594010daf0 con 0x7f59401032f0 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594816f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f594010d970 con 0x7f5940103cb0 2024-04-23T18:36:05.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594816f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f594010dc70 con 0x7f5940100280 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594570a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59401032f0 0x7f5940198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594570a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59401032f0 0x7f5940198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57690/0 (socket says 172.21.15.132:57690) 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594570a640 1 -- 172.21.15.132:0/647661527 learned_addr learned my addr 172.21.15.132:0/647661527 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f5945f0b640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5940100280 0x7f59401981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594670c640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5940103cb0 0x7f594019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594570a640 1 -- 172.21.15.132:0/647661527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5940100280 msgr2=0x7f59401981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594570a640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5940100280 0x7f59401981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594570a640 1 -- 172.21.15.132:0/647661527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5940103cb0 msgr2=0x7f594019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594570a640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5940103cb0 0x7f594019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.099+0000 7f594570a640 1 -- 172.21.15.132:0/647661527 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594019fda0 con 0x7f59401032f0 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.100+0000 7f594570a640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59401032f0 0x7f5940198710 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5930009a50 tx=0x7f5930031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.100+0000 7f5936ffd640 1 -- 172.21.15.132:0/647661527 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593003f030 con 0x7f59401032f0 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.100+0000 7f594816f640 1 -- 172.21.15.132:0/647661527 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59401a0030 con 0x7f59401032f0 2024-04-23T18:36:05.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.100+0000 7f5936ffd640 1 -- 172.21.15.132:0/647661527 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59300044b0 con 0x7f59401032f0 2024-04-23T18:36:05.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.100+0000 7f5936ffd640 1 -- 172.21.15.132:0/647661527 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930035070 con 0x7f59401032f0 2024-04-23T18:36:05.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.100+0000 7f594816f640 1 -- 172.21.15.132:0/647661527 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59401a0570 con 0x7f59401032f0 2024-04-23T18:36:05.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.101+0000 7f594816f640 1 -- 172.21.15.132:0/647661527 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59401094f0 con 0x7f59401032f0 2024-04-23T18:36:05.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.102+0000 7f5936ffd640 1 -- 172.21.15.132:0/647661527 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f5930042030 con 0x7f59401032f0 2024-04-23T18:36:05.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.102+0000 7f5936ffd640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f592c0a3a60 0x7f592c0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.103+0000 7f5945f0b640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f592c0a3a60 0x7f592c0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.103+0000 7f5936ffd640 1 -- 172.21.15.132:0/647661527 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f59300f07e0 con 0x7f59401032f0 2024-04-23T18:36:05.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.103+0000 7f5945f0b640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f592c0a3a60 0x7f592c0a5f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5938005fd0 tx=0x7f593800b410 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:05.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.107+0000 7f5936ffd640 1 -- 172.21.15.132:0/647661527 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59300bcd30 con 0x7f59401032f0 2024-04-23T18:36:05.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.312+0000 7f594816f640 1 -- 172.21.15.132:0/647661527 --> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5940062390 con 0x7f592c0a3a60 2024-04-23T18:36:05.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.319+0000 7f5936ffd640 1 -- 172.21.15.132:0/647661527 <== mgr.15366 v2:172.21.15.132:6800/3428544997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5940062390 con 0x7f592c0a3a60 2024-04-23T18:36:05.311 INFO:teuthology.orchestra.run.smithi132.stdout:/ 1m 6m3h 2024-04-23T18:36:05.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.321+0000 7f594816f640 1 -- 172.21.15.132:0/647661527 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f592c0a3a60 msgr2=0x7f592c0a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.321+0000 7f594816f640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f592c0a3a60 0x7f592c0a5f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5938005fd0 tx=0x7f593800b410 comp rx=0 tx=0).stop 2024-04-23T18:36:05.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.321+0000 7f594816f640 1 -- 172.21.15.132:0/647661527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59401032f0 msgr2=0x7f5940198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.321+0000 7f594816f640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59401032f0 0x7f5940198710 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5930009a50 tx=0x7f5930031a50 comp rx=0 tx=0).stop 2024-04-23T18:36:05.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.321+0000 7f594816f640 1 -- 172.21.15.132:0/647661527 shutdown_connections 2024-04-23T18:36:05.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.321+0000 7f594816f640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f592c0a3a60 0x7f592c0a5f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.321+0000 7f594816f640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5940103cb0 0x7f594019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.321+0000 7f594816f640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59401032f0 0x7f5940198710 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.321+0000 7f594816f640 1 --2- 172.21.15.132:0/647661527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5940100280 0x7f59401981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.321+0000 7f594816f640 1 -- 172.21.15.132:0/647661527 >> 172.21.15.132:0/647661527 conn(0x7f59400fbbb0 msgr2=0x7f59401066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:05.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.321+0000 7f594816f640 1 -- 172.21.15.132:0/647661527 shutdown_connections 2024-04-23T18:36:05.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.321+0000 7f594816f640 1 -- 172.21.15.132:0/647661527 wait complete. 2024-04-23T18:36:05.324 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-23T18:36:05.348 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-23T18:36:05.358 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-23T18:36:05.367 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-04-23T18:36:05.367 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-23T18:36:05.392 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-23T18:36:05.404 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:36:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.515+0000 7f5ec309a640 1 -- 172.21.15.132:0/2081416915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ebc102a10 msgr2=0x7f5ebc104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.515+0000 7f5ec309a640 1 --2- 172.21.15.132:0/2081416915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ebc102a10 0x7f5ebc104e00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009900 tx=0x7f5eb002a410 comp rx=0 tx=0).stop 2024-04-23T18:36:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.515+0000 7f5ec309a640 1 -- 172.21.15.132:0/2081416915 shutdown_connections 2024-04-23T18:36:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.515+0000 7f5ec309a640 1 --2- 172.21.15.132:0/2081416915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ebc107c70 0x7f5ebc10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.515+0000 7f5ec309a640 1 --2- 172.21.15.132:0/2081416915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ebc105340 0x7f5ebc107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.515+0000 7f5ec309a640 1 --2- 172.21.15.132:0/2081416915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ebc102a10 0x7f5ebc104e00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.515+0000 7f5ec309a640 1 -- 172.21.15.132:0/2081416915 >> 172.21.15.132:0/2081416915 conn(0x7f5ebc0fc400 msgr2=0x7f5ebc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.515+0000 7f5ec309a640 1 -- 172.21.15.132:0/2081416915 shutdown_connections 2024-04-23T18:36:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.515+0000 7f5ec309a640 1 -- 172.21.15.132:0/2081416915 wait complete. 2024-04-23T18:36:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec309a640 1 Processor -- start 2024-04-23T18:36:05.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec309a640 1 -- start start 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec309a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ebc102a10 0x7f5ebc102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec309a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ebc105340 0x7f5ebc1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec309a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ebc107c70 0x7f5ebc100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec309a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc10dac0 con 0x7f5ebc107c70 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec309a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc10d940 con 0x7f5ebc105340 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec309a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc10dc40 con 0x7f5ebc102a10 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec1637640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ebc107c70 0x7f5ebc100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec1637640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ebc107c70 0x7f5ebc100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57704/0 (socket says 172.21.15.132:57704) 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec1637640 1 -- 172.21.15.132:0/840803922 learned_addr learned my addr 172.21.15.132:0/840803922 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec0e36640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ebc102a10 0x7f5ebc102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ebbfff640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ebc105340 0x7f5ebc1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec1637640 1 -- 172.21.15.132:0/840803922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ebc102a10 msgr2=0x7f5ebc102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec1637640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ebc102a10 0x7f5ebc102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec1637640 1 -- 172.21.15.132:0/840803922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ebc105340 msgr2=0x7f5ebc1009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec1637640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ebc105340 0x7f5ebc1009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec1637640 1 -- 172.21.15.132:0/840803922 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc101790 con 0x7f5ebc107c70 2024-04-23T18:36:05.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.516+0000 7f5ec1637640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ebc107c70 0x7f5ebc100f00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400eb10 tx=0x7f5eb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:05.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.517+0000 7f5eb9ffb640 1 -- 172.21.15.132:0/840803922 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4004b20 con 0x7f5ebc107c70 2024-04-23T18:36:05.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.517+0000 7f5eb9ffb640 1 -- 172.21.15.132:0/840803922 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eb4004cc0 con 0x7f5ebc107c70 2024-04-23T18:36:05.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.517+0000 7f5ec309a640 1 -- 172.21.15.132:0/840803922 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc1a2310 con 0x7f5ebc107c70 2024-04-23T18:36:05.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.517+0000 7f5eb9ffb640 1 -- 172.21.15.132:0/840803922 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4010670 con 0x7f5ebc107c70 2024-04-23T18:36:05.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.517+0000 7f5ec309a640 1 -- 172.21.15.132:0/840803922 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc1a2810 con 0x7f5ebc107c70 2024-04-23T18:36:05.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.518+0000 7f5ec309a640 1 -- 172.21.15.132:0/840803922 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc0fdef0 con 0x7f5ebc107c70 2024-04-23T18:36:05.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.519+0000 7f5eb9ffb640 1 -- 172.21.15.132:0/840803922 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f5eb4005000 con 0x7f5ebc107c70 2024-04-23T18:36:05.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.520+0000 7f5eb9ffb640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f5eac0a39b0 0x7f5eac0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.520+0000 7f5ec0e36640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f5eac0a39b0 0x7f5eac0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.520+0000 7f5eb9ffb640 1 -- 172.21.15.132:0/840803922 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f5eb40d0930 con 0x7f5ebc107c70 2024-04-23T18:36:05.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.521+0000 7f5ec0e36640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f5eac0a39b0 0x7f5eac0a5e70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0000c00 tx=0x7f5eb00314a0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:05.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.524+0000 7f5eb9ffb640 1 -- 172.21.15.132:0/840803922 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5eb4053ff0 con 0x7f5ebc107c70 2024-04-23T18:36:05.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.539+0000 7f5eb9ffb640 1 -- 172.21.15.132:0/840803922 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7f5eb4099440 con 0x7f5ebc107c70 2024-04-23T18:36:05.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:05 smithi132 ceph-mon[29178]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:05.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:05 smithi132 ceph-mon[29178]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:36:05.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:05 smithi132 ceph-mon[29178]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:05.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:05 smithi132 ceph-mon[29178]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:05.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:05 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:05.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:05 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:05.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:05 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:05.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:05 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:05.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:05 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:05.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:05 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:05.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.724+0000 7f5ec309a640 1 -- 172.21.15.132:0/840803922 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ebc062b50 con 0x7f5ebc107c70 2024-04-23T18:36:05.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.725+0000 7f5eb9ffb640 1 -- 172.21.15.132:0/840803922 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23228 (secure 0 0 0) 0x7f5eb409d4d0 con 0x7f5ebc107c70 2024-04-23T18:36:05.718 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:05.718 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":91,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:36:02.717304+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/2124409853":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/3095429281":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/899407689":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1857586348":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:36:05.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.728+0000 7f5ec309a640 1 -- 172.21.15.132:0/840803922 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f5eac0a39b0 msgr2=0x7f5eac0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.728+0000 7f5ec309a640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f5eac0a39b0 0x7f5eac0a5e70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0000c00 tx=0x7f5eb00314a0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.728+0000 7f5ec309a640 1 -- 172.21.15.132:0/840803922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ebc107c70 msgr2=0x7f5ebc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.728+0000 7f5ec309a640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ebc107c70 0x7f5ebc100f00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400eb10 tx=0x7f5eb400c2d0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.728+0000 7f5ec309a640 1 -- 172.21.15.132:0/840803922 shutdown_connections 2024-04-23T18:36:05.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.728+0000 7f5ec309a640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f5eac0a39b0 0x7f5eac0a5e70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.728+0000 7f5ec309a640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ebc107c70 0x7f5ebc100f00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.728+0000 7f5ec309a640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ebc105340 0x7f5ebc1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.728+0000 7f5ec309a640 1 --2- 172.21.15.132:0/840803922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ebc102a10 0x7f5ebc102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.728+0000 7f5ec309a640 1 -- 172.21.15.132:0/840803922 >> 172.21.15.132:0/840803922 conn(0x7f5ebc0fc400 msgr2=0x7f5ebc103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:05.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.728+0000 7f5ec309a640 1 -- 172.21.15.132:0/840803922 shutdown_connections 2024-04-23T18:36:05.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.728+0000 7f5ec309a640 1 -- 172.21.15.132:0/840803922 wait complete. 2024-04-23T18:36:05.732 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:29:20.132451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:33:58.407521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:33:59.518206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:36:05.732 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:36:05.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:05 smithi157 ceph-mon[32580]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:05.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:05 smithi157 ceph-mon[32580]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:36:05.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:05 smithi157 ceph-mon[32580]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:05.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:05 smithi157 ceph-mon[32580]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:05.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:05 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:05.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:05 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:05.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:05 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:05.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:05 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:05.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:05 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:05.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:05 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:05.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.844+0000 7ff967c05640 1 -- 172.21.15.132:0/2464239454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff960102a10 msgr2=0x7ff960104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.844+0000 7ff967c05640 1 --2- 172.21.15.132:0/2464239454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff960102a10 0x7ff960104e00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009900 tx=0x7ff95c02a410 comp rx=0 tx=0).stop 2024-04-23T18:36:05.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.844+0000 7ff967c05640 1 -- 172.21.15.132:0/2464239454 shutdown_connections 2024-04-23T18:36:05.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.844+0000 7ff967c05640 1 --2- 172.21.15.132:0/2464239454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff960105340 0x7ff960107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.844+0000 7ff967c05640 1 --2- 172.21.15.132:0/2464239454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960107c70 0x7ff96010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.844+0000 7ff967c05640 1 --2- 172.21.15.132:0/2464239454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff960102a10 0x7ff960104e00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.844+0000 7ff967c05640 1 -- 172.21.15.132:0/2464239454 >> 172.21.15.132:0/2464239454 conn(0x7ff9600fc400 msgr2=0x7ff9600fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.844+0000 7ff967c05640 1 -- 172.21.15.132:0/2464239454 shutdown_connections 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.844+0000 7ff967c05640 1 -- 172.21.15.132:0/2464239454 wait complete. 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.844+0000 7ff967c05640 1 Processor -- start 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff967c05640 1 -- start start 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff967c05640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff960102a10 0x7ff960102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff967c05640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff960105340 0x7ff9601009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff967c05640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960107c70 0x7ff960100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff967c05640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff96010dac0 con 0x7ff960105340 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff967c05640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff96010d940 con 0x7ff960107c70 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff967c05640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff96010dc40 con 0x7ff960102a10 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff9651a0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff960105340 0x7ff9601009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff9651a0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff960105340 0x7ff9601009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57710/0 (socket says 172.21.15.132:57710) 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff9651a0640 1 -- 172.21.15.132:0/582618608 learned_addr learned my addr 172.21.15.132:0/582618608 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:05.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff9661a2640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960107c70 0x7ff960100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff9659a1640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff960102a10 0x7ff960102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff9651a0640 1 -- 172.21.15.132:0/582618608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff960102a10 msgr2=0x7ff960102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff9651a0640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff960102a10 0x7ff960102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff9651a0640 1 -- 172.21.15.132:0/582618608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960107c70 msgr2=0x7ff960100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:05.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff9651a0640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960107c70 0x7ff960100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:05.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff9651a0640 1 -- 172.21.15.132:0/582618608 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff960101790 con 0x7ff960105340 2024-04-23T18:36:05.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff9651a0640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff960105340 0x7ff9601009c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff95000e9c0 tx=0x7ff95000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:05.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff956ffd640 1 -- 172.21.15.132:0/582618608 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95000cd80 con 0x7ff960105340 2024-04-23T18:36:05.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff967c05640 1 -- 172.21.15.132:0/582618608 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9601a2360 con 0x7ff960105340 2024-04-23T18:36:05.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff956ffd640 1 -- 172.21.15.132:0/582618608 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9500054d0 con 0x7ff960105340 2024-04-23T18:36:05.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff956ffd640 1 -- 172.21.15.132:0/582618608 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950010430 con 0x7ff960105340 2024-04-23T18:36:05.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.845+0000 7ff967c05640 1 -- 172.21.15.132:0/582618608 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9601a28b0 con 0x7ff960105340 2024-04-23T18:36:05.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.846+0000 7ff967c05640 1 -- 172.21.15.132:0/582618608 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9600fdef0 con 0x7ff960105340 2024-04-23T18:36:05.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.848+0000 7ff956ffd640 1 -- 172.21.15.132:0/582618608 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7ff950004fb0 con 0x7ff960105340 2024-04-23T18:36:05.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.848+0000 7ff956ffd640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff94c0a3db0 0x7ff94c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:05.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.848+0000 7ff9659a1640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff94c0a3db0 0x7ff94c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:05.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.848+0000 7ff956ffd640 1 -- 172.21.15.132:0/582618608 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7ff9500d0fc0 con 0x7ff960105340 2024-04-23T18:36:05.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.849+0000 7ff9659a1640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff94c0a3db0 0x7ff94c0a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff95c000c00 tx=0x7ff95c0314a0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:05.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:05.852+0000 7ff956ffd640 1 -- 172.21.15.132:0/582618608 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff950053ff0 con 0x7ff960105340 2024-04-23T18:36:06.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:05 smithi148 ceph-mon[32311]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:06.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:05 smithi148 ceph-mon[32311]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:36:06.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:05 smithi148 ceph-mon[32311]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:06.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:05 smithi148 ceph-mon[32311]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:06.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:05 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:05 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:05 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:05 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:05 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:05 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.048+0000 7ff967c05640 1 -- 172.21.15.132:0/582618608 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff96002d810 con 0x7ff960105340 2024-04-23T18:36:06.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.048+0000 7ff956ffd640 1 -- 172.21.15.132:0/582618608 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7ff95009d4e0 con 0x7ff960105340 2024-04-23T18:36:06.041 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-04-23T18:36:06.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.051+0000 7ff967c05640 1 -- 172.21.15.132:0/582618608 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff94c0a3db0 msgr2=0x7ff94c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.051+0000 7ff967c05640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff94c0a3db0 0x7ff94c0a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff95c000c00 tx=0x7ff95c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.051+0000 7ff967c05640 1 -- 172.21.15.132:0/582618608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff960105340 msgr2=0x7ff9601009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.051+0000 7ff967c05640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff960105340 0x7ff9601009c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff95000e9c0 tx=0x7ff95000ee80 comp rx=0 tx=0).stop 2024-04-23T18:36:06.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.051+0000 7ff967c05640 1 -- 172.21.15.132:0/582618608 shutdown_connections 2024-04-23T18:36:06.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.051+0000 7ff967c05640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff94c0a3db0 0x7ff94c0a6270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.051+0000 7ff967c05640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960107c70 0x7ff960100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.051+0000 7ff967c05640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff960105340 0x7ff9601009c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.051+0000 7ff967c05640 1 --2- 172.21.15.132:0/582618608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff960102a10 0x7ff960102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.051+0000 7ff967c05640 1 -- 172.21.15.132:0/582618608 >> 172.21.15.132:0/582618608 conn(0x7ff9600fc400 msgr2=0x7ff9601048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:06.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.051+0000 7ff967c05640 1 -- 172.21.15.132:0/582618608 shutdown_connections 2024-04-23T18:36:06.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.051+0000 7ff967c05640 1 -- 172.21.15.132:0/582618608 wait complete. 2024-04-23T18:36:06.054 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:36:06.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 -- 172.21.15.132:0/2288922074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcedc102a10 msgr2=0x7fcedc104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 --2- 172.21.15.132:0/2288922074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcedc102a10 0x7fcedc104e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fced0009860 tx=0x7fced002a340 comp rx=0 tx=0).stop 2024-04-23T18:36:06.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 -- 172.21.15.132:0/2288922074 shutdown_connections 2024-04-23T18:36:06.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 --2- 172.21.15.132:0/2288922074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcedc105340 0x7fcedc107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 --2- 172.21.15.132:0/2288922074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcedc107c70 0x7fcedc10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 --2- 172.21.15.132:0/2288922074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcedc102a10 0x7fcedc104e00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 -- 172.21.15.132:0/2288922074 >> 172.21.15.132:0/2288922074 conn(0x7fcedc0fc400 msgr2=0x7fcedc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 -- 172.21.15.132:0/2288922074 shutdown_connections 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 -- 172.21.15.132:0/2288922074 wait complete. 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 Processor -- start 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 -- start start 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcedc102a10 0x7fcedc102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcedc105340 0x7fcedc1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcedc107c70 0x7fcedc100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc10dac0 con 0x7fcedc102a10 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc10d940 con 0x7fcedc105340 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.174+0000 7fcee2ada640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc10dc40 con 0x7fcedc107c70 2024-04-23T18:36:06.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcee0876640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcedc102a10 0x7fcedc102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcee0876640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcedc102a10 0x7fcedc102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57730/0 (socket says 172.21.15.132:57730) 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcee0876640 1 -- 172.21.15.132:0/3908655059 learned_addr learned my addr 172.21.15.132:0/3908655059 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcedbfff640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcedc105340 0x7fcedc1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcee1077640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcedc107c70 0x7fcedc100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcee0876640 1 -- 172.21.15.132:0/3908655059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcedc107c70 msgr2=0x7fcedc100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcee0876640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcedc107c70 0x7fcedc100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcee0876640 1 -- 172.21.15.132:0/3908655059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcedc105340 msgr2=0x7fcedc1009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcee0876640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcedc105340 0x7fcedc1009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcee0876640 1 -- 172.21.15.132:0/3908655059 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc1017c0 con 0x7fcedc102a10 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcedbfff640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcedc105340 0x7fcedc1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcee0876640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcedc102a10 0x7fcedc102330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fced0005940 tx=0x7fced002ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fced9ffb640 1 -- 172.21.15.132:0/3908655059 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0040020 con 0x7fcedc102a10 2024-04-23T18:36:06.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcee2ada640 1 -- 172.21.15.132:0/3908655059 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcedc1a2300 con 0x7fcedc102a10 2024-04-23T18:36:06.171 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.175+0000 7fcee2ada640 1 -- 172.21.15.132:0/3908655059 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcedc1a2810 con 0x7fcedc102a10 2024-04-23T18:36:06.171 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.176+0000 7fced9ffb640 1 -- 172.21.15.132:0/3908655059 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced0030e50 con 0x7fcedc102a10 2024-04-23T18:36:06.171 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.176+0000 7fced9ffb640 1 -- 172.21.15.132:0/3908655059 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0037040 con 0x7fcedc102a10 2024-04-23T18:36:06.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.177+0000 7fcee2ada640 1 -- 172.21.15.132:0/3908655059 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec00052c0 con 0x7fcedc102a10 2024-04-23T18:36:06.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.178+0000 7fced9ffb640 1 -- 172.21.15.132:0/3908655059 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7fced0043020 con 0x7fcedc102a10 2024-04-23T18:36:06.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.178+0000 7fced9ffb640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcecc0a3db0 0x7fcecc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.179+0000 7fced9ffb640 1 -- 172.21.15.132:0/3908655059 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fced00f0290 con 0x7fcedc102a10 2024-04-23T18:36:06.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.179+0000 7fcedbfff640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcecc0a3db0 0x7fcecc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.179+0000 7fcedbfff640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcecc0a3db0 0x7fcecc0a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcedc1020a0 tx=0x7fcec8005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:06.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.182+0000 7fced9ffb640 1 -- 172.21.15.132:0/3908655059 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fced004a030 con 0x7fcedc102a10 2024-04-23T18:36:06.393 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.401+0000 7fcee2ada640 1 -- 172.21.15.132:0/3908655059 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcec0005880 con 0x7fcedc102a10 2024-04-23T18:36:06.393 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.401+0000 7fced9ffb640 1 -- 172.21.15.132:0/3908655059 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fced00bc8c0 con 0x7fcedc102a10 2024-04-23T18:36:06.393 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-04-23T18:36:06.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.403+0000 7fcee2ada640 1 -- 172.21.15.132:0/3908655059 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcecc0a3db0 msgr2=0x7fcecc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.403+0000 7fcee2ada640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcecc0a3db0 0x7fcecc0a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcedc1020a0 tx=0x7fcec8005f90 comp rx=0 tx=0).stop 2024-04-23T18:36:06.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.403+0000 7fcee2ada640 1 -- 172.21.15.132:0/3908655059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcedc102a10 msgr2=0x7fcedc102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.403+0000 7fcee2ada640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcedc102a10 0x7fcedc102330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fced0005940 tx=0x7fced002ae20 comp rx=0 tx=0).stop 2024-04-23T18:36:06.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.403+0000 7fcee2ada640 1 -- 172.21.15.132:0/3908655059 shutdown_connections 2024-04-23T18:36:06.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.403+0000 7fcee2ada640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcecc0a3db0 0x7fcecc0a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.403+0000 7fcee2ada640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcedc107c70 0x7fcedc100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.403+0000 7fcee2ada640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcedc105340 0x7fcedc1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.403+0000 7fcee2ada640 1 --2- 172.21.15.132:0/3908655059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcedc102a10 0x7fcedc102330 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.403+0000 7fcee2ada640 1 -- 172.21.15.132:0/3908655059 >> 172.21.15.132:0/3908655059 conn(0x7fcedc0fc400 msgr2=0x7fcedc103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:06.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.403+0000 7fcee2ada640 1 -- 172.21.15.132:0/3908655059 shutdown_connections 2024-04-23T18:36:06.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.403+0000 7fcee2ada640 1 -- 172.21.15.132:0/3908655059 wait complete. 2024-04-23T18:36:06.406 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:36:06.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.518+0000 7f8428f5d640 1 -- 172.21.15.132:0/2772401029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8424105340 msgr2=0x7f8424107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.518+0000 7f8428f5d640 1 --2- 172.21.15.132:0/2772401029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8424105340 0x7f8424107730 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8418009900 tx=0x7f841802a410 comp rx=0 tx=0).stop 2024-04-23T18:36:06.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.518+0000 7f8428f5d640 1 -- 172.21.15.132:0/2772401029 shutdown_connections 2024-04-23T18:36:06.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.518+0000 7f8428f5d640 1 --2- 172.21.15.132:0/2772401029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107c70 0x7f842410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.518+0000 7f8428f5d640 1 --2- 172.21.15.132:0/2772401029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8424105340 0x7f8424107730 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.518+0000 7f8428f5d640 1 --2- 172.21.15.132:0/2772401029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8424102a10 0x7f8424104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.518+0000 7f8428f5d640 1 -- 172.21.15.132:0/2772401029 >> 172.21.15.132:0/2772401029 conn(0x7f84240fc400 msgr2=0x7f84240fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:06.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.519+0000 7f8428f5d640 1 -- 172.21.15.132:0/2772401029 shutdown_connections 2024-04-23T18:36:06.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.519+0000 7f8428f5d640 1 -- 172.21.15.132:0/2772401029 wait complete. 2024-04-23T18:36:06.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.519+0000 7f8428f5d640 1 Processor -- start 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.519+0000 7f8428f5d640 1 -- start start 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.519+0000 7f8428f5d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8424102a10 0x7f8424198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.519+0000 7f8428f5d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8424105340 0x7f8424198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.519+0000 7f8428f5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107c70 0x7f842419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.519+0000 7f8428f5d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f842410daa0 con 0x7f8424102a10 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.519+0000 7f8428f5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f842410d920 con 0x7f8424107c70 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.519+0000 7f8428f5d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f842410dc20 con 0x7f8424105340 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f842259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8424102a10 0x7f8424198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8421d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8424105340 0x7f8424198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8422d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107c70 0x7f842419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8421d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8424105340 0x7f8424198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.132:40960/0 (socket says 172.21.15.132:40960) 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8422d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107c70 0x7f842419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.132:35788/0 (socket says 172.21.15.132:35788) 2024-04-23T18:36:06.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8422d9d640 1 -- 172.21.15.132:0/2917454541 learned_addr learned my addr 172.21.15.132:0/2917454541 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f842259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8424102a10 0x7f8424198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57734/0 (socket says 172.21.15.132:57734) 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8422d9d640 1 -- 172.21.15.132:0/2917454541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8424105340 msgr2=0x7f8424198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8422d9d640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8424105340 0x7f8424198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8422d9d640 1 -- 172.21.15.132:0/2917454541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8424102a10 msgr2=0x7f8424198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8422d9d640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8424102a10 0x7f8424198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8422d9d640 1 -- 172.21.15.132:0/2917454541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842419fe60 con 0x7f8424107c70 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8421d9b640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8424105340 0x7f8424198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f842259c640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8424102a10 0x7f8424198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8422d9d640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107c70 0x7f842419f7f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f841c009fd0 tx=0x7f841c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f84137fe640 1 -- 172.21.15.132:0/2917454541 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c017070 con 0x7f8424107c70 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8428f5d640 1 -- 172.21.15.132:0/2917454541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84241a0150 con 0x7f8424107c70 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f84137fe640 1 -- 172.21.15.132:0/2917454541 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f841c003e20 con 0x7f8424107c70 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f84137fe640 1 -- 172.21.15.132:0/2917454541 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c010460 con 0x7f8424107c70 2024-04-23T18:36:06.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.520+0000 7f8428f5d640 1 -- 172.21.15.132:0/2917454541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84241a0620 con 0x7f8424107c70 2024-04-23T18:36:06.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.521+0000 7f8428f5d640 1 -- 172.21.15.132:0/2917454541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84240fdef0 con 0x7f8424107c70 2024-04-23T18:36:06.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.523+0000 7f84137fe640 1 -- 172.21.15.132:0/2917454541 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7f841c003970 con 0x7f8424107c70 2024-04-23T18:36:06.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.523+0000 7f84137fe640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f840c0a3db0 0x7f840c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.523+0000 7f842259c640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f840c0a3db0 0x7f840c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.523+0000 7f84137fe640 1 -- 172.21.15.132:0/2917454541 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f841c0d06c0 con 0x7f8424107c70 2024-04-23T18:36:06.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.524+0000 7f842259c640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f840c0a3db0 0x7f840c0a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f841400bfd0 tx=0x7f841400b390 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:06.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.527+0000 7f84137fe640 1 -- 172.21.15.132:0/2917454541 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f841c09c530 con 0x7f8424107c70 2024-04-23T18:36:06.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.721+0000 7f8428f5d640 1 -- 172.21.15.132:0/2917454541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8424062b50 con 0x7f8424107c70 2024-04-23T18:36:06.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.722+0000 7f84137fe640 1 -- 172.21.15.132:0/2917454541 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f841c0a0830 con 0x7f8424107c70 2024-04-23T18:36:06.714 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-04-23T18:36:06.714 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:36:06.715 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:06.715 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:06.715 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-23T18:36:06.715 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/840803922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:36:06.715 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.715 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.715 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.715 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.715 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.715 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.715 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.715 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/582618608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:36:06.716 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:06 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3908655059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:36:06.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.725+0000 7f8428f5d640 1 -- 172.21.15.132:0/2917454541 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f840c0a3db0 msgr2=0x7f840c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.725+0000 7f8428f5d640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f840c0a3db0 0x7f840c0a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f841400bfd0 tx=0x7f841400b390 comp rx=0 tx=0).stop 2024-04-23T18:36:06.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.725+0000 7f8428f5d640 1 -- 172.21.15.132:0/2917454541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107c70 msgr2=0x7f842419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.725+0000 7f8428f5d640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107c70 0x7f842419f7f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f841c009fd0 tx=0x7f841c00b260 comp rx=0 tx=0).stop 2024-04-23T18:36:06.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.725+0000 7f8428f5d640 1 -- 172.21.15.132:0/2917454541 shutdown_connections 2024-04-23T18:36:06.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.725+0000 7f8428f5d640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f840c0a3db0 0x7f840c0a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.725+0000 7f8428f5d640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8424105340 0x7f8424198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.725+0000 7f8428f5d640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107c70 0x7f842419f7f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.725+0000 7f8428f5d640 1 --2- 172.21.15.132:0/2917454541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8424102a10 0x7f8424198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.725+0000 7f8428f5d640 1 -- 172.21.15.132:0/2917454541 >> 172.21.15.132:0/2917454541 conn(0x7f84240fc400 msgr2=0x7f8424103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:06.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.725+0000 7f8428f5d640 1 -- 172.21.15.132:0/2917454541 shutdown_connections 2024-04-23T18:36:06.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.725+0000 7f8428f5d640 1 -- 172.21.15.132:0/2917454541 wait complete. 2024-04-23T18:36:06.729 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:36:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:36:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-23T18:36:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/840803922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:36:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/582618608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:36:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:06.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:36:06.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:36:06.792 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:06 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3908655059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:36:06.832 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.837+0000 7f397f319640 1 -- 172.21.15.132:0/814391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3978100e90 msgr2=0x7f39781012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.832 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.837+0000 7f397f319640 1 --2- 172.21.15.132:0/814391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3978100e90 0x7f39781012b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3974009900 tx=0x7f397402a410 comp rx=0 tx=0).stop 2024-04-23T18:36:06.832 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.837+0000 7f397f319640 1 -- 172.21.15.132:0/814391 shutdown_connections 2024-04-23T18:36:06.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.837+0000 7f397f319640 1 --2- 172.21.15.132:0/814391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39781029a0 0x7f39781092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.837+0000 7f397f319640 1 --2- 172.21.15.132:0/814391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3978101fe0 0x7f3978102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.837+0000 7f397f319640 1 --2- 172.21.15.132:0/814391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3978100e90 0x7f39781012b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.837+0000 7f397f319640 1 -- 172.21.15.132:0/814391 >> 172.21.15.132:0/814391 conn(0x7f39780fc400 msgr2=0x7f39780fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:06.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397f319640 1 -- 172.21.15.132:0/814391 shutdown_connections 2024-04-23T18:36:06.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397f319640 1 -- 172.21.15.132:0/814391 wait complete. 2024-04-23T18:36:06.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397f319640 1 Processor -- start 2024-04-23T18:36:06.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397f319640 1 -- start start 2024-04-23T18:36:06.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397f319640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3978100e90 0x7f3978069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397f319640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3978101fe0 0x7f397819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397f319640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39781029a0 0x7f39781a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397f319640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f397810c9e0 con 0x7f3978100e90 2024-04-23T18:36:06.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397f319640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f397810c860 con 0x7f3978101fe0 2024-04-23T18:36:06.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397f319640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f397810cb60 con 0x7f39781029a0 2024-04-23T18:36:06.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397d0b5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3978100e90 0x7f3978069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397d0b5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3978100e90 0x7f3978069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57750/0 (socket says 172.21.15.132:57750) 2024-04-23T18:36:06.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397d0b5640 1 -- 172.21.15.132:0/3045867145 learned_addr learned my addr 172.21.15.132:0/3045867145 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:06.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.838+0000 7f397d8b6640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39781029a0 0x7f39781a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f397c8b4640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3978101fe0 0x7f397819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f397d0b5640 1 -- 172.21.15.132:0/3045867145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39781029a0 msgr2=0x7f39781a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f397d0b5640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39781029a0 0x7f39781a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f397d0b5640 1 -- 172.21.15.132:0/3045867145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3978101fe0 msgr2=0x7f397819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:06.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f397d0b5640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3978101fe0 0x7f397819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:06.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f397d0b5640 1 -- 172.21.15.132:0/3045867145 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39781a1fe0 con 0x7f3978100e90 2024-04-23T18:36:06.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f397d8b6640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39781029a0 0x7f39781a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:06.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f397c8b4640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3978101fe0 0x7f397819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:06.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f397d0b5640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3978100e90 0x7f3978069c50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3974005940 tx=0x7f3974031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:06.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f396e7fc640 1 -- 172.21.15.132:0/3045867145 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397403f030 con 0x7f3978100e90 2024-04-23T18:36:06.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f397f319640 1 -- 172.21.15.132:0/3045867145 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39781a2210 con 0x7f3978100e90 2024-04-23T18:36:06.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f396e7fc640 1 -- 172.21.15.132:0/3045867145 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3974003aa0 con 0x7f3978100e90 2024-04-23T18:36:06.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f396e7fc640 1 -- 172.21.15.132:0/3045867145 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974035070 con 0x7f3978100e90 2024-04-23T18:36:06.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.839+0000 7f397f319640 1 -- 172.21.15.132:0/3045867145 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39781a2750 con 0x7f3978100e90 2024-04-23T18:36:06.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.840+0000 7f397f319640 1 -- 172.21.15.132:0/3045867145 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39781081f0 con 0x7f3978100e90 2024-04-23T18:36:06.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.842+0000 7f396e7fc640 1 -- 172.21.15.132:0/3045867145 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7f3974042030 con 0x7f3978100e90 2024-04-23T18:36:06.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.843+0000 7f396e7fc640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f39640a3e60 0x7f39640a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:06.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.843+0000 7f397c8b4640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f39640a3e60 0x7f39640a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:06.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.843+0000 7f396e7fc640 1 -- 172.21.15.132:0/3045867145 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f39740f0f30 con 0x7f3978100e90 2024-04-23T18:36:06.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.843+0000 7f397c8b4640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f39640a3e60 0x7f39640a6320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f39780699c0 tx=0x7f396800a5e0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:06.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:06.847+0000 7f396e7fc640 1 -- 172.21.15.132:0/3045867145 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f39740bd450 con 0x7f3978100e90 2024-04-23T18:36:07.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/840803922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:07.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:07.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:07.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:07.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/582618608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:36:07.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:07.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:07.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:07.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:07.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:36:07.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:36:07.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:06 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3908655059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:36:07.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.034+0000 7f397f319640 1 -- 172.21.15.132:0/3045867145 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f397802d810 con 0x7f3978100e90 2024-04-23T18:36:07.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.036+0000 7f396e7fc640 1 -- 172.21.15.132:0/3045867145 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23228 (secure 0 0 0) 0x7f39740bcd50 con 0x7f3978100e90 2024-04-23T18:36:07.028 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:07.028 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":91,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:36:02.717304+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/2124409853":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/3095429281":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/899407689":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1857586348":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:36:07.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.038+0000 7f397f319640 1 -- 172.21.15.132:0/3045867145 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f39640a3e60 msgr2=0x7f39640a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.038+0000 7f397f319640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f39640a3e60 0x7f39640a6320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f39780699c0 tx=0x7f396800a5e0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.038+0000 7f397f319640 1 -- 172.21.15.132:0/3045867145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3978100e90 msgr2=0x7f3978069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.038+0000 7f397f319640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3978100e90 0x7f3978069c50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3974005940 tx=0x7f3974031a50 comp rx=0 tx=0).stop 2024-04-23T18:36:07.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.038+0000 7f397f319640 1 -- 172.21.15.132:0/3045867145 shutdown_connections 2024-04-23T18:36:07.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.038+0000 7f397f319640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f39640a3e60 0x7f39640a6320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.038+0000 7f397f319640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39781029a0 0x7f39781a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.038+0000 7f397f319640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3978101fe0 0x7f397819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.038+0000 7f397f319640 1 --2- 172.21.15.132:0/3045867145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3978100e90 0x7f3978069c50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.038+0000 7f397f319640 1 -- 172.21.15.132:0/3045867145 >> 172.21.15.132:0/3045867145 conn(0x7f39780fc400 msgr2=0x7f39781053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:07.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.038+0000 7f397f319640 1 -- 172.21.15.132:0/3045867145 shutdown_connections 2024-04-23T18:36:07.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.038+0000 7f397f319640 1 -- 172.21.15.132:0/3045867145 wait complete. 2024-04-23T18:36:07.042 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:29:20.132451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:33:58.407521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:33:59.518206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:36:07.042 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:36:07.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.154+0000 7f94e152d640 1 -- 172.21.15.132:0/393287492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94dc107270 msgr2=0x7f94dc10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.154+0000 7f94e152d640 1 --2- 172.21.15.132:0/393287492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94dc107270 0x7f94dc10ee40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009ca0 tx=0x7f94d402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.154+0000 7f94e152d640 1 -- 172.21.15.132:0/393287492 shutdown_connections 2024-04-23T18:36:07.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.154+0000 7f94e152d640 1 --2- 172.21.15.132:0/393287492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc06b1d0 0x7f94dc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.154+0000 7f94e152d640 1 --2- 172.21.15.132:0/393287492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94dc107270 0x7f94dc10ee40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.154+0000 7f94e152d640 1 --2- 172.21.15.132:0/393287492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94dc06a820 0x7f94dc06ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.154+0000 7f94e152d640 1 -- 172.21.15.132:0/393287492 >> 172.21.15.132:0/393287492 conn(0x7f94dc077eb0 msgr2=0x7f94dc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:07.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.154+0000 7f94e152d640 1 -- 172.21.15.132:0/393287492 shutdown_connections 2024-04-23T18:36:07.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.154+0000 7f94e152d640 1 -- 172.21.15.132:0/393287492 wait complete. 2024-04-23T18:36:07.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.154+0000 7f94e152d640 1 Processor -- start 2024-04-23T18:36:07.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94e152d640 1 -- start start 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94e152d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94dc06a820 0x7f94dc1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94e152d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94dc107270 0x7f94dc1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94e152d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc19a680 0x7f94dc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94e152d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1121d0 con 0x7f94dc107270 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94e152d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc112050 con 0x7f94dc06a820 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94e152d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc112350 con 0x7f94dc19a680 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94da7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94dc107270 0x7f94dc1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94da7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94dc107270 0x7f94dc1a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57758/0 (socket says 172.21.15.132:57758) 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94da7fc640 1 -- 172.21.15.132:0/3380793441 learned_addr learned my addr 172.21.15.132:0/3380793441 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94db7fe640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc19a680 0x7f94dc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94daffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94dc06a820 0x7f94dc1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94db7fe640 1 -- 172.21.15.132:0/3380793441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94dc06a820 msgr2=0x7f94dc1a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94db7fe640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94dc06a820 0x7f94dc1a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94db7fe640 1 -- 172.21.15.132:0/3380793441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94dc107270 msgr2=0x7f94dc1a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94db7fe640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94dc107270 0x7f94dc1a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94db7fe640 1 -- 172.21.15.132:0/3380793441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc19b280 con 0x7f94dc19a680 2024-04-23T18:36:07.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94da7fc640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94dc107270 0x7f94dc1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:07.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.155+0000 7f94daffd640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94dc06a820 0x7f94dc1a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:07.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.156+0000 7f94db7fe640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc19a680 0x7f94dc19ab00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f94d402ade0 tx=0x7f94d40319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:07.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.156+0000 7f94c3fff640 1 -- 172.21.15.132:0/3380793441 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4003bf0 con 0x7f94dc19a680 2024-04-23T18:36:07.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.156+0000 7f94e152d640 1 -- 172.21.15.132:0/3380793441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc19b510 con 0x7f94dc19a680 2024-04-23T18:36:07.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.156+0000 7f94c3fff640 1 -- 172.21.15.132:0/3380793441 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94d4003d90 con 0x7f94dc19a680 2024-04-23T18:36:07.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.156+0000 7f94c3fff640 1 -- 172.21.15.132:0/3380793441 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d402fbd0 con 0x7f94dc19a680 2024-04-23T18:36:07.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.156+0000 7f94e152d640 1 -- 172.21.15.132:0/3380793441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc1a7700 con 0x7f94dc19a680 2024-04-23T18:36:07.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.157+0000 7f94e152d640 1 -- 172.21.15.132:0/3380793441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94dc103670 con 0x7f94dc19a680 2024-04-23T18:36:07.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.158+0000 7f94c3fff640 1 -- 172.21.15.132:0/3380793441 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7f94d402fd70 con 0x7f94dc19a680 2024-04-23T18:36:07.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.159+0000 7f94c3fff640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f94c40a3db0 0x7f94c40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:07.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.159+0000 7f94daffd640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f94c40a3db0 0x7f94c40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:07.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.159+0000 7f94c3fff640 1 -- 172.21.15.132:0/3380793441 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f94d40f1ee0 con 0x7f94dc19a680 2024-04-23T18:36:07.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.159+0000 7f94daffd640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f94c40a3db0 0x7f94c40a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f94d000afb0 tx=0x7f94d000a2d0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:07.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.163+0000 7f94c3fff640 1 -- 172.21.15.132:0/3380793441 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94d40be4b0 con 0x7f94dc19a680 2024-04-23T18:36:07.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.351+0000 7f94e152d640 1 -- 172.21.15.132:0/3380793441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f94dc06b1d0 con 0x7f94dc19a680 2024-04-23T18:36:07.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.352+0000 7f94c3fff640 1 -- 172.21.15.132:0/3380793441 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f94d40098b0 con 0x7f94dc19a680 2024-04-23T18:36:07.343 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-04-23T18:36:07.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.354+0000 7f94e152d640 1 -- 172.21.15.132:0/3380793441 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f94c40a3db0 msgr2=0x7f94c40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.354+0000 7f94e152d640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f94c40a3db0 0x7f94c40a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f94d000afb0 tx=0x7f94d000a2d0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.354+0000 7f94e152d640 1 -- 172.21.15.132:0/3380793441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc19a680 msgr2=0x7f94dc19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.354+0000 7f94e152d640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc19a680 0x7f94dc19ab00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f94d402ade0 tx=0x7f94d40319f0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.354+0000 7f94e152d640 1 -- 172.21.15.132:0/3380793441 shutdown_connections 2024-04-23T18:36:07.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.354+0000 7f94e152d640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f94c40a3db0 0x7f94c40a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.354+0000 7f94e152d640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94dc19a680 0x7f94dc19ab00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.354+0000 7f94e152d640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94dc107270 0x7f94dc1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.354+0000 7f94e152d640 1 --2- 172.21.15.132:0/3380793441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94dc06a820 0x7f94dc1a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.354+0000 7f94e152d640 1 -- 172.21.15.132:0/3380793441 >> 172.21.15.132:0/3380793441 conn(0x7f94dc077eb0 msgr2=0x7f94dc10d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:07.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.354+0000 7f94e152d640 1 -- 172.21.15.132:0/3380793441 shutdown_connections 2024-04-23T18:36:07.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.354+0000 7f94e152d640 1 -- 172.21.15.132:0/3380793441 wait complete. 2024-04-23T18:36:07.357 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:36:07.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.474+0000 7fcf2dbad640 1 -- 172.21.15.132:0/4013763125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf28103ae0 msgr2=0x7fcf2810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.474+0000 7fcf2dbad640 1 --2- 172.21.15.132:0/4013763125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf28103ae0 0x7fcf2810bfc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009900 tx=0x7fcf1c02a510 comp rx=0 tx=0).stop 2024-04-23T18:36:07.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.474+0000 7fcf2dbad640 1 -- 172.21.15.132:0/4013763125 shutdown_connections 2024-04-23T18:36:07.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.474+0000 7fcf2dbad640 1 --2- 172.21.15.132:0/4013763125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf2810c500 0x7fcf2810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.474+0000 7fcf2dbad640 1 --2- 172.21.15.132:0/4013763125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf28103ae0 0x7fcf2810bfc0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.474+0000 7fcf2dbad640 1 --2- 172.21.15.132:0/4013763125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf281031c0 0x7fcf281035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.474+0000 7fcf2dbad640 1 -- 172.21.15.132:0/4013763125 >> 172.21.15.132:0/4013763125 conn(0x7fcf28077fd0 msgr2=0x7fcf280783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:07.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.474+0000 7fcf2dbad640 1 -- 172.21.15.132:0/4013763125 shutdown_connections 2024-04-23T18:36:07.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.474+0000 7fcf2dbad640 1 -- 172.21.15.132:0/4013763125 wait complete. 2024-04-23T18:36:07.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.475+0000 7fcf2dbad640 1 Processor -- start 2024-04-23T18:36:07.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.475+0000 7fcf2dbad640 1 -- start start 2024-04-23T18:36:07.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.475+0000 7fcf2dbad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281031c0 0x7fcf2819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:07.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.475+0000 7fcf2dbad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf28103ae0 0x7fcf2819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:07.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.475+0000 7fcf2dbad640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf2810c500 0x7fcf281985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:07.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.475+0000 7fcf2dbad640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcf28112190 con 0x7fcf2810c500 2024-04-23T18:36:07.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.475+0000 7fcf2dbad640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcf28112010 con 0x7fcf28103ae0 2024-04-23T18:36:07.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.475+0000 7fcf2dbad640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf28112310 con 0x7fcf281031c0 2024-04-23T18:36:07.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf27fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf2810c500 0x7fcf281985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:07.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf27fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf2810c500 0x7fcf281985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57768/0 (socket says 172.21.15.132:57768) 2024-04-23T18:36:07.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf27fff640 1 -- 172.21.15.132:0/28674201 learned_addr learned my addr 172.21.15.132:0/28674201 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:07.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf26ffd640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf28103ae0 0x7fcf2819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:07.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf277fe640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281031c0 0x7fcf2819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:07.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf27fff640 1 -- 172.21.15.132:0/28674201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281031c0 msgr2=0x7fcf2819e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf27fff640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281031c0 0x7fcf2819e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf27fff640 1 -- 172.21.15.132:0/28674201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf28103ae0 msgr2=0x7fcf2819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf27fff640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf28103ae0 0x7fcf2819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf27fff640 1 -- 172.21.15.132:0/28674201 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf28198d50 con 0x7fcf2810c500 2024-04-23T18:36:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf277fe640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281031c0 0x7fcf2819e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf27fff640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf2810c500 0x7fcf281985d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000ea60 tx=0x7fcf2000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf24ff9640 1 -- 172.21.15.132:0/28674201 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000ce40 con 0x7fcf2810c500 2024-04-23T18:36:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf24ff9640 1 -- 172.21.15.132:0/28674201 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf20005020 con 0x7fcf2810c500 2024-04-23T18:36:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf2dbad640 1 -- 172.21.15.132:0/28674201 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf28199040 con 0x7fcf2810c500 2024-04-23T18:36:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf24ff9640 1 -- 172.21.15.132:0/28674201 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20010670 con 0x7fcf2810c500 2024-04-23T18:36:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.476+0000 7fcf2dbad640 1 -- 172.21.15.132:0/28674201 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf281a52e0 con 0x7fcf2810c500 2024-04-23T18:36:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.477+0000 7fcf2dbad640 1 -- 172.21.15.132:0/28674201 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf280684e0 con 0x7fcf2810c500 2024-04-23T18:36:07.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.479+0000 7fcf24ff9640 1 -- 172.21.15.132:0/28674201 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7fcf200054b0 con 0x7fcf2810c500 2024-04-23T18:36:07.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.479+0000 7fcf24ff9640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcf100a3db0 0x7fcf100a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:07.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.479+0000 7fcf277fe640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcf100a3db0 0x7fcf100a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:07.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.479+0000 7fcf24ff9640 1 -- 172.21.15.132:0/28674201 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fcf200d10f0 con 0x7fcf2810c500 2024-04-23T18:36:07.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.480+0000 7fcf277fe640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcf100a3db0 0x7fcf100a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009c20 tx=0x7fcf18005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:07.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.483+0000 7fcf24ff9640 1 -- 172.21.15.132:0/28674201 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf20053ff0 con 0x7fcf2810c500 2024-04-23T18:36:07.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.563+0000 7fcf24ff9640 1 -- 172.21.15.132:0/28674201 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fcf20099520 con 0x7fcf2810c500 2024-04-23T18:36:07.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-23T18:36:07.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: Unable to set osd_memory_target on smithi148 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:36:07.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-23T18:36:07.612 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: Unable to set osd_memory_target on smithi157 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:36:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi132 to 176.2M 2024-04-23T18:36:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: Unable to set osd_memory_target on smithi132 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-04-23T18:36:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: Updating smithi132:/etc/ceph/ceph.conf 2024-04-23T18:36:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-23T18:36:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-23T18:36:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:36:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: Updating smithi132:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:36:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:36:07.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2917454541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:36:07.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3045867145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:36:07.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3380793441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:36:07.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:36:07.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:36:07.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:36:07.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:07 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.670+0000 7fcf2dbad640 1 -- 172.21.15.132:0/28674201 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcf28107680 con 0x7fcf2810c500 2024-04-23T18:36:07.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.670+0000 7fcf24ff9640 1 -- 172.21.15.132:0/28674201 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fcf200c65c0 con 0x7fcf2810c500 2024-04-23T18:36:07.663 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-04-23T18:36:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.672+0000 7fcf2dbad640 1 -- 172.21.15.132:0/28674201 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcf100a3db0 msgr2=0x7fcf100a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.672+0000 7fcf2dbad640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcf100a3db0 0x7fcf100a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009c20 tx=0x7fcf18005f90 comp rx=0 tx=0).stop 2024-04-23T18:36:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.672+0000 7fcf2dbad640 1 -- 172.21.15.132:0/28674201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf2810c500 msgr2=0x7fcf281985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.673+0000 7fcf2dbad640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf2810c500 0x7fcf281985d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000ea60 tx=0x7fcf2000ef20 comp rx=0 tx=0).stop 2024-04-23T18:36:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.673+0000 7fcf2dbad640 1 -- 172.21.15.132:0/28674201 shutdown_connections 2024-04-23T18:36:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.673+0000 7fcf2dbad640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcf100a3db0 0x7fcf100a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.673+0000 7fcf2dbad640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf2810c500 0x7fcf281985d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.673+0000 7fcf2dbad640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf28103ae0 0x7fcf2819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.673+0000 7fcf2dbad640 1 --2- 172.21.15.132:0/28674201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281031c0 0x7fcf2819e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.673+0000 7fcf2dbad640 1 -- 172.21.15.132:0/28674201 >> 172.21.15.132:0/28674201 conn(0x7fcf28077fd0 msgr2=0x7fcf280ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:07.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.673+0000 7fcf2dbad640 1 -- 172.21.15.132:0/28674201 shutdown_connections 2024-04-23T18:36:07.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.673+0000 7fcf2dbad640 1 -- 172.21.15.132:0/28674201 wait complete. 2024-04-23T18:36:07.676 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:36:07.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.786+0000 7f507a8e8640 1 -- 172.21.15.132:0/3558742433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5074101a40 msgr2=0x7f5074105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.786+0000 7f507a8e8640 1 --2- 172.21.15.132:0/3558742433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5074101a40 0x7f5074105a80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5068009900 tx=0x7f506802a410 comp rx=0 tx=0).stop 2024-04-23T18:36:07.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.786+0000 7f507a8e8640 1 -- 172.21.15.132:0/3558742433 shutdown_connections 2024-04-23T18:36:07.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.786+0000 7f507a8e8640 1 --2- 172.21.15.132:0/3558742433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5074106160 0x7f507410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.786+0000 7f507a8e8640 1 --2- 172.21.15.132:0/3558742433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5074101a40 0x7f5074105a80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.786+0000 7f507a8e8640 1 --2- 172.21.15.132:0/3558742433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5074101090 0x7f5074101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.786+0000 7f507a8e8640 1 -- 172.21.15.132:0/3558742433 >> 172.21.15.132:0/3558742433 conn(0x7f5074077f50 msgr2=0x7f5074078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.786+0000 7f507a8e8640 1 -- 172.21.15.132:0/3558742433 shutdown_connections 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.786+0000 7f507a8e8640 1 -- 172.21.15.132:0/3558742433 wait complete. 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.787+0000 7f507a8e8640 1 Processor -- start 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.787+0000 7f507a8e8640 1 -- start start 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.787+0000 7f507a8e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5074101090 0x7f507410e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.787+0000 7f507a8e8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5074106160 0x7f507410ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.787+0000 7f507a8e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f507410f140 0x7f50741a6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.787+0000 7f507a8e8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5074112200 con 0x7f5074106160 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.787+0000 7f507a8e8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5074112080 con 0x7f5074101090 2024-04-23T18:36:07.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.787+0000 7f507a8e8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5074112380 con 0x7f507410f140 2024-04-23T18:36:07.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.787+0000 7f50737fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5074106160 0x7f507410ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:07.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f50737fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5074106160 0x7f507410ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57790/0 (socket says 172.21.15.132:57790) 2024-04-23T18:36:07.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f50737fe640 1 -- 172.21.15.132:0/777435506 learned_addr learned my addr 172.21.15.132:0/777435506 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:07.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f5073fff640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5074101090 0x7f507410e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:07.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f5078e85640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f507410f140 0x7f50741a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:07.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f50737fe640 1 -- 172.21.15.132:0/777435506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f507410f140 msgr2=0x7f50741a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f50737fe640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f507410f140 0x7f50741a6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f50737fe640 1 -- 172.21.15.132:0/777435506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5074101090 msgr2=0x7f507410e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f50737fe640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5074101090 0x7f507410e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f50737fe640 1 -- 172.21.15.132:0/777435506 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50741a7090 con 0x7f5074106160 2024-04-23T18:36:07.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f50737fe640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5074106160 0x7f507410ec00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5068032fd0 tx=0x7f5068004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:07.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f50717fa640 1 -- 172.21.15.132:0/777435506 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506803f030 con 0x7f5074106160 2024-04-23T18:36:07.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f50717fa640 1 -- 172.21.15.132:0/777435506 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5068004490 con 0x7f5074106160 2024-04-23T18:36:07.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f507a8e8640 1 -- 172.21.15.132:0/777435506 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50741a7320 con 0x7f5074106160 2024-04-23T18:36:07.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f50717fa640 1 -- 172.21.15.132:0/777435506 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068035070 con 0x7f5074106160 2024-04-23T18:36:07.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.788+0000 7f507a8e8640 1 -- 172.21.15.132:0/777435506 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50741a7860 con 0x7f5074106160 2024-04-23T18:36:07.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.789+0000 7f507a8e8640 1 -- 172.21.15.132:0/777435506 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5074068d70 con 0x7f5074106160 2024-04-23T18:36:07.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.790+0000 7f50717fa640 1 -- 172.21.15.132:0/777435506 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f5068042030 con 0x7f5074106160 2024-04-23T18:36:07.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.791+0000 7f50717fa640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f505c0a3ef0 0x7f505c0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:07.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.791+0000 7f5073fff640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f505c0a3ef0 0x7f505c0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:07.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.791+0000 7f50717fa640 1 -- 172.21.15.132:0/777435506 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f50680f0d60 con 0x7f5074106160 2024-04-23T18:36:07.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.791+0000 7f5073fff640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f505c0a3ef0 0x7f505c0a63b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5064005d00 tx=0x7f506400b410 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:07.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.796+0000 7f50717fa640 1 -- 172.21.15.132:0/777435506 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50680bcc20 con 0x7f5074106160 2024-04-23T18:36:07.916 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-23T18:36:07.916 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: Unable to set osd_memory_target on smithi148 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:36:07.916 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: Unable to set osd_memory_target on smithi157 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi132 to 176.2M 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: Unable to set osd_memory_target on smithi132 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: Updating smithi132:/etc/ceph/ceph.conf 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: Updating smithi132:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2917454541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3045867145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.918 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.918 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.918 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.918 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3380793441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:36:07.918 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:36:07.918 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:36:07.918 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:36:07.918 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:07 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:07.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.973+0000 7f507a8e8640 1 -- 172.21.15.132:0/777435506 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5074109390 con 0x7f5074106160 2024-04-23T18:36:07.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.974+0000 7f50717fa640 1 -- 172.21.15.132:0/777435506 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f50680c0f20 con 0x7f5074106160 2024-04-23T18:36:07.966 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-04-23T18:36:07.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.976+0000 7f507a8e8640 1 -- 172.21.15.132:0/777435506 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f505c0a3ef0 msgr2=0x7f505c0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.976+0000 7f507a8e8640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f505c0a3ef0 0x7f505c0a63b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5064005d00 tx=0x7f506400b410 comp rx=0 tx=0).stop 2024-04-23T18:36:07.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.976+0000 7f507a8e8640 1 -- 172.21.15.132:0/777435506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5074106160 msgr2=0x7f507410ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:07.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.976+0000 7f507a8e8640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5074106160 0x7f507410ec00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5068032fd0 tx=0x7f5068004270 comp rx=0 tx=0).stop 2024-04-23T18:36:07.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.976+0000 7f507a8e8640 1 -- 172.21.15.132:0/777435506 shutdown_connections 2024-04-23T18:36:07.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.976+0000 7f507a8e8640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f505c0a3ef0 0x7f505c0a63b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.976+0000 7f507a8e8640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f507410f140 0x7f50741a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.976+0000 7f507a8e8640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5074106160 0x7f507410ec00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.976+0000 7f507a8e8640 1 --2- 172.21.15.132:0/777435506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5074101090 0x7f507410e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:07.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.976+0000 7f507a8e8640 1 -- 172.21.15.132:0/777435506 >> 172.21.15.132:0/777435506 conn(0x7f5074077f50 msgr2=0x7f50740ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:07.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.976+0000 7f507a8e8640 1 -- 172.21.15.132:0/777435506 shutdown_connections 2024-04-23T18:36:07.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:07.976+0000 7f507a8e8640 1 -- 172.21.15.132:0/777435506 wait complete. 2024-04-23T18:36:07.979 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-23T18:36:07.979 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:36:08.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-23T18:36:08.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: Unable to set osd_memory_target on smithi148 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:36:08.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-23T18:36:08.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: Unable to set osd_memory_target on smithi157 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:36:08.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi132 to 176.2M 2024-04-23T18:36:08.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: Unable to set osd_memory_target on smithi132 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-04-23T18:36:08.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: Updating smithi132:/etc/ceph/ceph.conf 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: Updating smithi132:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2917454541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3045867145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:08.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:08.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3380793441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:36:08.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:36:08.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:36:08.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:36:08.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:07 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:36:08.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.086+0000 7fecf9f4a640 1 -- 172.21.15.132:0/932588924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf4107280 msgr2=0x7fecf410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.086+0000 7fecf9f4a640 1 --2- 172.21.15.132:0/932588924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf4107280 0x7fecf410ee50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fecec009ca0 tx=0x7fecec02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.087+0000 7fecf9f4a640 1 -- 172.21.15.132:0/932588924 shutdown_connections 2024-04-23T18:36:08.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.087+0000 7fecf9f4a640 1 --2- 172.21.15.132:0/932588924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf4102b60 0x7fecf4106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.087+0000 7fecf9f4a640 1 --2- 172.21.15.132:0/932588924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf4107280 0x7fecf410ee50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.087+0000 7fecf9f4a640 1 --2- 172.21.15.132:0/932588924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf41021b0 0x7fecf4102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.087+0000 7fecf9f4a640 1 -- 172.21.15.132:0/932588924 >> 172.21.15.132:0/932588924 conn(0x7fecf40fd9f0 msgr2=0x7fecf40ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.087+0000 7fecf9f4a640 1 -- 172.21.15.132:0/932588924 shutdown_connections 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.087+0000 7fecf9f4a640 1 -- 172.21.15.132:0/932588924 wait complete. 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.087+0000 7fecf9f4a640 1 Processor -- start 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.087+0000 7fecf9f4a640 1 -- start start 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.087+0000 7fecf9f4a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf41021b0 0x7fecf4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.087+0000 7fecf9f4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf4107280 0x7fecf4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf9f4a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf419cfd0 0x7fecf419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf9f4a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fecf41121f0 con 0x7fecf41021b0 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf9f4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fecf4112070 con 0x7fecf4107280 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf9f4a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fecf4112370 con 0x7fecf419cfd0 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf37fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf41021b0 0x7fecf4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf37fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf41021b0 0x7fecf4198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57806/0 (socket says 172.21.15.132:57806) 2024-04-23T18:36:08.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf37fe640 1 -- 172.21.15.132:0/3423285348 learned_addr learned my addr 172.21.15.132:0/3423285348 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:08.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf37fe640 1 -- 172.21.15.132:0/3423285348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf419cfd0 msgr2=0x7fecf419d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf37fe640 1 --2- 172.21.15.132:0/3423285348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf419cfd0 0x7fecf419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf37fe640 1 -- 172.21.15.132:0/3423285348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf4107280 msgr2=0x7fecf4198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf37fe640 1 --2- 172.21.15.132:0/3423285348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf4107280 0x7fecf4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf37fe640 1 -- 172.21.15.132:0/3423285348 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf419dac0 con 0x7fecf41021b0 2024-04-23T18:36:08.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf37fe640 1 --2- 172.21.15.132:0/3423285348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf41021b0 0x7fecf4198710 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fece400c920 tx=0x7fece400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:08.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf0ff9640 1 -- 172.21.15.132:0/3423285348 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4007c70 con 0x7fecf41021b0 2024-04-23T18:36:08.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf0ff9640 1 -- 172.21.15.132:0/3423285348 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece4007e10 con 0x7fecf41021b0 2024-04-23T18:36:08.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf9f4a640 1 -- 172.21.15.132:0/3423285348 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf419ddb0 con 0x7fecf41021b0 2024-04-23T18:36:08.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf0ff9640 1 -- 172.21.15.132:0/3423285348 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4018820 con 0x7fecf41021b0 2024-04-23T18:36:08.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.088+0000 7fecf9f4a640 1 -- 172.21.15.132:0/3423285348 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf41a55a0 con 0x7fecf41021b0 2024-04-23T18:36:08.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.089+0000 7fecf9f4a640 1 -- 172.21.15.132:0/3423285348 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf4068d70 con 0x7fecf41021b0 2024-04-23T18:36:08.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.090+0000 7fecf0ff9640 1 -- 172.21.15.132:0/3423285348 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fece400f450 con 0x7fecf41021b0 2024-04-23T18:36:08.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.091+0000 7fecf0ff9640 1 --2- 172.21.15.132:0/3423285348 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fecdc0a3e40 0x7fecdc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:08.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.091+0000 7fecf2ffd640 1 --2- 172.21.15.132:0/3423285348 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fecdc0a3e40 0x7fecdc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:08.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.091+0000 7fecf0ff9640 1 -- 172.21.15.132:0/3423285348 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fece40d0c90 con 0x7fecf41021b0 2024-04-23T18:36:08.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.091+0000 7fecf2ffd640 1 --2- 172.21.15.132:0/3423285348 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fecdc0a3e40 0x7fecdc0a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fece8005d80 tx=0x7fece8005c90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:08.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.095+0000 7fecf0ff9640 1 -- 172.21.15.132:0/3423285348 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fece4014030 con 0x7fecf41021b0 2024-04-23T18:36:08.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.273+0000 7fecf9f4a640 1 -- 172.21.15.132:0/3423285348 --> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fecf4102bb0 con 0x7fecdc0a3e40 2024-04-23T18:36:08.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.280+0000 7fecf0ff9640 1 -- 172.21.15.132:0/3423285348 <== mgr.15366 v2:172.21.15.132:6800/3428544997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258101 (secure 0 0 0) 0x7fecf4102bb0 con 0x7fecdc0a3e40 2024-04-23T18:36:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:08.277 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-04-23T18:36:08.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.284+0000 7fecf9f4a640 1 -- 172.21.15.132:0/3423285348 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fecdc0a3e40 msgr2=0x7fecdc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.284+0000 7fecf9f4a640 1 --2- 172.21.15.132:0/3423285348 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fecdc0a3e40 0x7fecdc0a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fece8005d80 tx=0x7fece8005c90 comp rx=0 tx=0).stop 2024-04-23T18:36:08.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.284+0000 7fecf9f4a640 1 -- 172.21.15.132:0/3423285348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf41021b0 msgr2=0x7fecf4198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.284+0000 7fecf9f4a640 1 --2- 172.21.15.132:0/3423285348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf41021b0 0x7fecf4198710 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fece400c920 tx=0x7fece400cde0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.284+0000 7fecf9f4a640 1 -- 172.21.15.132:0/3423285348 shutdown_connections 2024-04-23T18:36:08.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.284+0000 7fecf9f4a640 1 --2- 172.21.15.132:0/3423285348 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fecdc0a3e40 0x7fecdc0a6300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.284+0000 7fecf9f4a640 1 --2- 172.21.15.132:0/3423285348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf419cfd0 0x7fecf419d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.284+0000 7fecf9f4a640 1 --2- 172.21.15.132:0/3423285348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf4107280 0x7fecf4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.284+0000 7fecf9f4a640 1 --2- 172.21.15.132:0/3423285348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf41021b0 0x7fecf4198710 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.284+0000 7fecf9f4a640 1 -- 172.21.15.132:0/3423285348 >> 172.21.15.132:0/3423285348 conn(0x7fecf40fd9f0 msgr2=0x7fecf40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:08.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.284+0000 7fecf9f4a640 1 -- 172.21.15.132:0/3423285348 shutdown_connections 2024-04-23T18:36:08.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.284+0000 7fecf9f4a640 1 -- 172.21.15.132:0/3423285348 wait complete. 2024-04-23T18:36:08.287 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-23T18:36:06.735487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1184798,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1455,"num_read_kb":888,"num_write":554,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353260,"kb_used_data":12388,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588564,"statfs":{"total":1151940427776,"available":1151578689536,"internally_reserved":0,"allocated":12685312,"data_stored":9264187,"data_compressed":356910,"data_compressed_allocated":483328,"data_compressed_original":3710976,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990809"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734914+0000","last_change":"2024-04-23T18:34:01.231204+0000","last_active":"2024-04-23T18:36:02.734914+0000","last_peered":"2024-04-23T18:36:02.734914+0000","last_clean":"2024-04-23T18:36:02.734914+0000","last_became_active":"2024-04-23T18:34:01.230836+0000","last_became_peered":"2024-04-23T18:34:01.230836+0000","last_unstale":"2024-04-23T18:36:02.734914+0000","last_undegraded":"2024-04-23T18:36:02.734914+0000","last_fullsized":"2024-04-23T18:36:02.734914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:57.744807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765309+0000","last_change":"2024-04-23T18:34:01.225833+0000","last_active":"2024-04-23T18:36:02.765309+0000","last_peered":"2024-04-23T18:36:02.765309+0000","last_clean":"2024-04-23T18:36:02.765309+0000","last_became_active":"2024-04-23T18:34:01.225098+0000","last_became_peered":"2024-04-23T18:34:01.225098+0000","last_unstale":"2024-04-23T18:36:02.765309+0000","last_undegraded":"2024-04-23T18:36:02.765309+0000","last_fullsized":"2024-04-23T18:36:02.765309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:20.720137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765240+0000","last_change":"2024-04-23T18:34:01.235269+0000","last_active":"2024-04-23T18:36:02.765240+0000","last_peered":"2024-04-23T18:36:02.765240+0000","last_clean":"2024-04-23T18:36:02.765240+0000","last_became_active":"2024-04-23T18:34:01.234921+0000","last_became_peered":"2024-04-23T18:34:01.234921+0000","last_unstale":"2024-04-23T18:36:02.765240+0000","last_undegraded":"2024-04-23T18:36:02.765240+0000","last_fullsized":"2024-04-23T18:36:02.765240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:06:20.645693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.725703+0000","last_change":"2024-04-23T18:34:01.233316+0000","last_active":"2024-04-23T18:36:02.725703+0000","last_peered":"2024-04-23T18:36:02.725703+0000","last_clean":"2024-04-23T18:36:02.725703+0000","last_became_active":"2024-04-23T18:34:01.231636+0000","last_became_peered":"2024-04-23T18:34:01.231636+0000","last_unstale":"2024-04-23T18:36:02.725703+0000","last_undegraded":"2024-04-23T18:36:02.725703+0000","last_fullsized":"2024-04-23T18:36:02.725703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:15.837129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765118+0000","last_change":"2024-04-23T18:34:01.228073+0000","last_active":"2024-04-23T18:36:02.765118+0000","last_peered":"2024-04-23T18:36:02.765118+0000","last_clean":"2024-04-23T18:36:02.765118+0000","last_became_active":"2024-04-23T18:34:01.227701+0000","last_became_peered":"2024-04-23T18:34:01.227701+0000","last_unstale":"2024-04-23T18:36:02.765118+0000","last_undegraded":"2024-04-23T18:36:02.765118+0000","last_fullsized":"2024-04-23T18:36:02.765118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:17:26.556498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732650+0000","last_change":"2024-04-23T18:34:55.258233+0000","last_active":"2024-04-23T18:36:02.732650+0000","last_peered":"2024-04-23T18:36:02.732650+0000","last_clean":"2024-04-23T18:36:02.732650+0000","last_became_active":"2024-04-23T18:34:55.257836+0000","last_became_peered":"2024-04-23T18:34:55.257836+0000","last_unstale":"2024-04-23T18:36:02.732650+0000","last_undegraded":"2024-04-23T18:36:02.732650+0000","last_fullsized":"2024-04-23T18:36:02.732650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:20.279369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765125+0000","last_change":"2024-04-23T18:34:55.260737+0000","last_active":"2024-04-23T18:36:02.765125+0000","last_peered":"2024-04-23T18:36:02.765125+0000","last_clean":"2024-04-23T18:36:02.765125+0000","last_became_active":"2024-04-23T18:34:55.260231+0000","last_became_peered":"2024-04-23T18:34:55.260231+0000","last_unstale":"2024-04-23T18:36:02.765125+0000","last_undegraded":"2024-04-23T18:36:02.765125+0000","last_fullsized":"2024-04-23T18:36:02.765125+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:45.292690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.721402+0000","last_change":"2024-04-23T18:34:55.259963+0000","last_active":"2024-04-23T18:36:02.721402+0000","last_peered":"2024-04-23T18:36:02.721402+0000","last_clean":"2024-04-23T18:36:02.721402+0000","last_became_active":"2024-04-23T18:34:55.259342+0000","last_became_peered":"2024-04-23T18:34:55.259342+0000","last_unstale":"2024-04-23T18:36:02.721402+0000","last_undegraded":"2024-04-23T18:36:02.721402+0000","last_fullsized":"2024-04-23T18:36:02.721402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:42:36.612809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765111+0000","last_change":"2024-04-23T18:34:01.234094+0000","last_active":"2024-04-23T18:36:02.765111+0000","last_peered":"2024-04-23T18:36:02.765111+0000","last_clean":"2024-04-23T18:36:02.765111+0000","last_became_active":"2024-04-23T18:34:01.232499+0000","last_became_peered":"2024-04-23T18:34:01.232499+0000","last_unstale":"2024-04-23T18:36:02.765111+0000","last_undegraded":"2024-04-23T18:36:02.765111+0000","last_fullsized":"2024-04-23T18:36:02.765111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:33.004245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732991+0000","last_change":"2024-04-23T18:34:01.229175+0000","last_active":"2024-04-23T18:36:02.732991+0000","last_peered":"2024-04-23T18:36:02.732991+0000","last_clean":"2024-04-23T18:36:02.732991+0000","last_became_active":"2024-04-23T18:34:01.228837+0000","last_became_peered":"2024-04-23T18:34:01.228837+0000","last_unstale":"2024-04-23T18:36:02.732991+0000","last_undegraded":"2024-04-23T18:36:02.732991+0000","last_fullsized":"2024-04-23T18:36:02.732991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:57.625110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765546+0000","last_change":"2024-04-23T18:34:01.235557+0000","last_active":"2024-04-23T18:36:02.765546+0000","last_peered":"2024-04-23T18:36:02.765546+0000","last_clean":"2024-04-23T18:36:02.765546+0000","last_became_active":"2024-04-23T18:34:01.235184+0000","last_became_peered":"2024-04-23T18:34:01.235184+0000","last_unstale":"2024-04-23T18:36:02.765546+0000","last_undegraded":"2024-04-23T18:36:02.765546+0000","last_fullsized":"2024-04-23T18:36:02.765546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:37:27.138041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.725500+0000","last_change":"2024-04-23T18:34:01.234070+0000","last_active":"2024-04-23T18:36:02.725500+0000","last_peered":"2024-04-23T18:36:02.725500+0000","last_clean":"2024-04-23T18:36:02.725500+0000","last_became_active":"2024-04-23T18:34:01.233727+0000","last_became_peered":"2024-04-23T18:34:01.233727+0000","last_unstale":"2024-04-23T18:36:02.725500+0000","last_undegraded":"2024-04-23T18:36:02.725500+0000","last_fullsized":"2024-04-23T18:36:02.725500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:59:16.489819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731502+0000","last_change":"2024-04-23T18:34:01.228256+0000","last_active":"2024-04-23T18:36:02.731502+0000","last_peered":"2024-04-23T18:36:02.731502+0000","last_clean":"2024-04-23T18:36:02.731502+0000","last_became_active":"2024-04-23T18:34:01.224400+0000","last_became_peered":"2024-04-23T18:34:01.224400+0000","last_unstale":"2024-04-23T18:36:02.731502+0000","last_undegraded":"2024-04-23T18:36:02.731502+0000","last_fullsized":"2024-04-23T18:36:02.731502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:37:28.401695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764642+0000","last_change":"2024-04-23T18:34:01.227250+0000","last_active":"2024-04-23T18:36:02.764642+0000","last_peered":"2024-04-23T18:36:02.764642+0000","last_clean":"2024-04-23T18:36:02.764642+0000","last_became_active":"2024-04-23T18:34:01.226937+0000","last_became_peered":"2024-04-23T18:34:01.226937+0000","last_unstale":"2024-04-23T18:36:02.764642+0000","last_undegraded":"2024-04-23T18:36:02.764642+0000","last_fullsized":"2024-04-23T18:36:02.764642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:26:20.801691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764582+0000","last_change":"2024-04-23T18:34:01.232351+0000","last_active":"2024-04-23T18:36:02.764582+0000","last_peered":"2024-04-23T18:36:02.764582+0000","last_clean":"2024-04-23T18:36:02.764582+0000","last_became_active":"2024-04-23T18:34:01.231987+0000","last_became_peered":"2024-04-23T18:34:01.231987+0000","last_unstale":"2024-04-23T18:36:02.764582+0000","last_undegraded":"2024-04-23T18:36:02.764582+0000","last_fullsized":"2024-04-23T18:36:02.764582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:50.886496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764465+0000","last_change":"2024-04-23T18:34:01.232196+0000","last_active":"2024-04-23T18:36:02.764465+0000","last_peered":"2024-04-23T18:36:02.764465+0000","last_clean":"2024-04-23T18:36:02.764465+0000","last_became_active":"2024-04-23T18:34:01.231746+0000","last_became_peered":"2024-04-23T18:34:01.231746+0000","last_unstale":"2024-04-23T18:36:02.764465+0000","last_undegraded":"2024-04-23T18:36:02.764465+0000","last_fullsized":"2024-04-23T18:36:02.764465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:58.888549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.735096+0000","last_change":"2024-04-23T18:34:01.231509+0000","last_active":"2024-04-23T18:36:02.735096+0000","last_peered":"2024-04-23T18:36:02.735096+0000","last_clean":"2024-04-23T18:36:02.735096+0000","last_became_active":"2024-04-23T18:34:01.231017+0000","last_became_peered":"2024-04-23T18:34:01.231017+0000","last_unstale":"2024-04-23T18:36:02.735096+0000","last_undegraded":"2024-04-23T18:36:02.735096+0000","last_fullsized":"2024-04-23T18:36:02.735096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:04:51.929462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.721702+0000","last_change":"2024-04-23T18:34:01.223285+0000","last_active":"2024-04-23T18:36:02.721702+0000","last_peered":"2024-04-23T18:36:02.721702+0000","last_clean":"2024-04-23T18:36:02.721702+0000","last_became_active":"2024-04-23T18:34:01.222962+0000","last_became_peered":"2024-04-23T18:34:01.222962+0000","last_unstale":"2024-04-23T18:36:02.721702+0000","last_undegraded":"2024-04-23T18:36:02.721702+0000","last_fullsized":"2024-04-23T18:36:02.721702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:18.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722773+0000","last_change":"2024-04-23T18:34:01.225922+0000","last_active":"2024-04-23T18:36:02.722773+0000","last_peered":"2024-04-23T18:36:02.722773+0000","last_clean":"2024-04-23T18:36:02.722773+0000","last_became_active":"2024-04-23T18:34:01.225235+0000","last_became_peered":"2024-04-23T18:34:01.225235+0000","last_unstale":"2024-04-23T18:36:02.722773+0000","last_undegraded":"2024-04-23T18:36:02.722773+0000","last_fullsized":"2024-04-23T18:36:02.722773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:33.480782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.699597+0000","last_change":"2024-04-23T18:34:00.224606+0000","last_active":"2024-04-23T18:36:03.699597+0000","last_peered":"2024-04-23T18:36:03.699597+0000","last_clean":"2024-04-23T18:36:03.699597+0000","last_became_active":"2024-04-23T18:34:00.224291+0000","last_became_peered":"2024-04-23T18:34:00.224291+0000","last_unstale":"2024-04-23T18:36:03.699597+0000","last_undegraded":"2024-04-23T18:36:03.699597+0000","last_fullsized":"2024-04-23T18:36:03.699597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:40.734704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765428+0000","last_change":"2024-04-23T18:34:01.235023+0000","last_active":"2024-04-23T18:36:02.765428+0000","last_peered":"2024-04-23T18:36:02.765428+0000","last_clean":"2024-04-23T18:36:02.765428+0000","last_became_active":"2024-04-23T18:34:01.234672+0000","last_became_peered":"2024-04-23T18:34:01.234672+0000","last_unstale":"2024-04-23T18:36:02.765428+0000","last_undegraded":"2024-04-23T18:36:02.765428+0000","last_fullsized":"2024-04-23T18:36:02.765428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:33.734343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765629+0000","last_change":"2024-04-23T18:34:00.232552+0000","last_active":"2024-04-23T18:36:02.765629+0000","last_peered":"2024-04-23T18:36:02.765629+0000","last_clean":"2024-04-23T18:36:02.765629+0000","last_became_active":"2024-04-23T18:34:00.232226+0000","last_became_peered":"2024-04-23T18:34:00.232226+0000","last_unstale":"2024-04-23T18:36:02.765629+0000","last_undegraded":"2024-04-23T18:36:02.765629+0000","last_fullsized":"2024-04-23T18:36:02.765629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.563757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765732+0000","last_change":"2024-04-23T18:34:01.236013+0000","last_active":"2024-04-23T18:36:02.765732+0000","last_peered":"2024-04-23T18:36:02.765732+0000","last_clean":"2024-04-23T18:36:02.765732+0000","last_became_active":"2024-04-23T18:34:01.235590+0000","last_became_peered":"2024-04-23T18:34:01.235590+0000","last_unstale":"2024-04-23T18:36:02.765732+0000","last_undegraded":"2024-04-23T18:36:02.765732+0000","last_fullsized":"2024-04-23T18:36:02.765732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:45.741648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'13","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765471+0000","last_change":"2024-04-23T18:34:55.298010+0000","last_active":"2024-04-23T18:36:02.765471+0000","last_peered":"2024-04-23T18:36:02.765471+0000","last_clean":"2024-04-23T18:36:02.765471+0000","last_became_active":"2024-04-23T18:34:55.288221+0000","last_became_peered":"2024-04-23T18:34:55.288221+0000","last_unstale":"2024-04-23T18:36:02.765471+0000","last_undegraded":"2024-04-23T18:36:02.765471+0000","last_fullsized":"2024-04-23T18:36:02.765471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:21:35.469676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764723+0000","last_change":"2024-04-23T18:34:01.230435+0000","last_active":"2024-04-23T18:36:02.764723+0000","last_peered":"2024-04-23T18:36:02.764723+0000","last_clean":"2024-04-23T18:36:02.764723+0000","last_became_active":"2024-04-23T18:34:01.230095+0000","last_became_peered":"2024-04-23T18:34:01.230095+0000","last_unstale":"2024-04-23T18:36:02.764723+0000","last_undegraded":"2024-04-23T18:36:02.764723+0000","last_fullsized":"2024-04-23T18:36:02.764723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:31.653254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765399+0000","last_change":"2024-04-23T18:34:00.227853+0000","last_active":"2024-04-23T18:36:02.765399+0000","last_peered":"2024-04-23T18:36:02.765399+0000","last_clean":"2024-04-23T18:36:02.765399+0000","last_became_active":"2024-04-23T18:34:00.226596+0000","last_became_peered":"2024-04-23T18:34:00.226596+0000","last_unstale":"2024-04-23T18:36:02.765399+0000","last_undegraded":"2024-04-23T18:36:02.765399+0000","last_fullsized":"2024-04-23T18:36:02.765399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:16:10.077969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765100+0000","last_change":"2024-04-23T18:34:01.235955+0000","last_active":"2024-04-23T18:36:02.765100+0000","last_peered":"2024-04-23T18:36:02.765100+0000","last_clean":"2024-04-23T18:36:02.765100+0000","last_became_active":"2024-04-23T18:34:01.235609+0000","last_became_peered":"2024-04-23T18:34:01.235609+0000","last_unstale":"2024-04-23T18:36:02.765100+0000","last_undegraded":"2024-04-23T18:36:02.765100+0000","last_fullsized":"2024-04-23T18:36:02.765100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:07:39.480213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764707+0000","last_change":"2024-04-23T18:34:00.226319+0000","last_active":"2024-04-23T18:36:02.764707+0000","last_peered":"2024-04-23T18:36:02.764707+0000","last_clean":"2024-04-23T18:36:02.764707+0000","last_became_active":"2024-04-23T18:34:00.225796+0000","last_became_peered":"2024-04-23T18:34:00.225796+0000","last_unstale":"2024-04-23T18:36:02.764707+0000","last_undegraded":"2024-04-23T18:36:02.764707+0000","last_fullsized":"2024-04-23T18:36:02.764707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:11:18.309181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765238+0000","last_change":"2024-04-23T18:34:01.227336+0000","last_active":"2024-04-23T18:36:02.765238+0000","last_peered":"2024-04-23T18:36:02.765238+0000","last_clean":"2024-04-23T18:36:02.765238+0000","last_became_active":"2024-04-23T18:34:01.226987+0000","last_became_peered":"2024-04-23T18:34:01.226987+0000","last_unstale":"2024-04-23T18:36:02.765238+0000","last_undegraded":"2024-04-23T18:36:02.765238+0000","last_fullsized":"2024-04-23T18:36:02.765238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:22:30.058673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732492+0000","last_change":"2024-04-23T18:34:55.294322+0000","last_active":"2024-04-23T18:36:02.732492+0000","last_peered":"2024-04-23T18:36:02.732492+0000","last_clean":"2024-04-23T18:36:02.732492+0000","last_became_active":"2024-04-23T18:34:55.288855+0000","last_became_peered":"2024-04-23T18:34:55.288855+0000","last_unstale":"2024-04-23T18:36:02.732492+0000","last_undegraded":"2024-04-23T18:36:02.732492+0000","last_fullsized":"2024-04-23T18:36:02.732492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:07:50.097696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764793+0000","last_change":"2024-04-23T18:34:55.287339+0000","last_active":"2024-04-23T18:36:02.764793+0000","last_peered":"2024-04-23T18:36:02.764793+0000","last_clean":"2024-04-23T18:36:02.764793+0000","last_became_active":"2024-04-23T18:34:55.287062+0000","last_became_peered":"2024-04-23T18:34:55.287062+0000","last_unstale":"2024-04-23T18:36:02.764793+0000","last_undegraded":"2024-04-23T18:36:02.764793+0000","last_fullsized":"2024-04-23T18:36:02.764793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:44.864600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731488+0000","last_change":"2024-04-23T18:34:00.227595+0000","last_active":"2024-04-23T18:36:02.731488+0000","last_peered":"2024-04-23T18:36:02.731488+0000","last_clean":"2024-04-23T18:36:02.731488+0000","last_became_active":"2024-04-23T18:34:00.226830+0000","last_became_peered":"2024-04-23T18:34:00.226830+0000","last_unstale":"2024-04-23T18:36:02.731488+0000","last_undegraded":"2024-04-23T18:36:02.731488+0000","last_fullsized":"2024-04-23T18:36:02.731488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:20.661984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764758+0000","last_change":"2024-04-23T18:34:01.225053+0000","last_active":"2024-04-23T18:36:02.764758+0000","last_peered":"2024-04-23T18:36:02.764758+0000","last_clean":"2024-04-23T18:36:02.764758+0000","last_became_active":"2024-04-23T18:34:01.224723+0000","last_became_peered":"2024-04-23T18:34:01.224723+0000","last_unstale":"2024-04-23T18:36:02.764758+0000","last_undegraded":"2024-04-23T18:36:02.764758+0000","last_fullsized":"2024-04-23T18:36:02.764758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:01:51.486691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764506+0000","last_change":"2024-04-23T18:34:00.224783+0000","last_active":"2024-04-23T18:36:02.764506+0000","last_peered":"2024-04-23T18:36:02.764506+0000","last_clean":"2024-04-23T18:36:02.764506+0000","last_became_active":"2024-04-23T18:34:00.224436+0000","last_became_peered":"2024-04-23T18:34:00.224436+0000","last_unstale":"2024-04-23T18:36:02.764506+0000","last_undegraded":"2024-04-23T18:36:02.764506+0000","last_fullsized":"2024-04-23T18:36:02.764506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:33.829188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.721558+0000","last_change":"2024-04-23T18:34:01.224283+0000","last_active":"2024-04-23T18:36:02.721558+0000","last_peered":"2024-04-23T18:36:02.721558+0000","last_clean":"2024-04-23T18:36:02.721558+0000","last_became_active":"2024-04-23T18:34:01.223949+0000","last_became_peered":"2024-04-23T18:34:01.223949+0000","last_unstale":"2024-04-23T18:36:02.721558+0000","last_undegraded":"2024-04-23T18:36:02.721558+0000","last_fullsized":"2024-04-23T18:36:02.721558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:50.560687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":268,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.690843+0000","last_change":"2024-04-23T18:34:00.218453+0000","last_active":"2024-04-23T18:36:03.690843+0000","last_peered":"2024-04-23T18:36:03.690843+0000","last_clean":"2024-04-23T18:36:03.690843+0000","last_became_active":"2024-04-23T18:34:00.218036+0000","last_became_peered":"2024-04-23T18:34:00.218036+0000","last_unstale":"2024-04-23T18:36:03.690843+0000","last_undegraded":"2024-04-23T18:36:03.690843+0000","last_fullsized":"2024-04-23T18:36:03.690843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:49.485185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764932+0000","last_change":"2024-04-23T18:34:01.231143+0000","last_active":"2024-04-23T18:36:02.764932+0000","last_peered":"2024-04-23T18:36:02.764932+0000","last_clean":"2024-04-23T18:36:02.764932+0000","last_became_active":"2024-04-23T18:34:01.230812+0000","last_became_peered":"2024-04-23T18:34:01.230812+0000","last_unstale":"2024-04-23T18:36:02.764932+0000","last_undegraded":"2024-04-23T18:36:02.764932+0000","last_fullsized":"2024-04-23T18:36:02.764932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:28.704807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765595+0000","last_change":"2024-04-23T18:34:00.235711+0000","last_active":"2024-04-23T18:36:02.765595+0000","last_peered":"2024-04-23T18:36:02.765595+0000","last_clean":"2024-04-23T18:36:02.765595+0000","last_became_active":"2024-04-23T18:34:00.235333+0000","last_became_peered":"2024-04-23T18:34:00.235333+0000","last_unstale":"2024-04-23T18:36:02.765595+0000","last_undegraded":"2024-04-23T18:36:02.765595+0000","last_fullsized":"2024-04-23T18:36:02.765595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:51.320744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765599+0000","last_change":"2024-04-23T18:34:01.238291+0000","last_active":"2024-04-23T18:36:02.765599+0000","last_peered":"2024-04-23T18:36:02.765599+0000","last_clean":"2024-04-23T18:36:02.765599+0000","last_became_active":"2024-04-23T18:34:01.237972+0000","last_became_peered":"2024-04-23T18:34:01.237972+0000","last_unstale":"2024-04-23T18:36:02.765599+0000","last_undegraded":"2024-04-23T18:36:02.765599+0000","last_fullsized":"2024-04-23T18:36:02.765599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:36:17.108170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765280+0000","last_change":"2024-04-23T18:34:00.227249+0000","last_active":"2024-04-23T18:36:02.765280+0000","last_peered":"2024-04-23T18:36:02.765280+0000","last_clean":"2024-04-23T18:36:02.765280+0000","last_became_active":"2024-04-23T18:34:00.226854+0000","last_became_peered":"2024-04-23T18:34:00.226854+0000","last_unstale":"2024-04-23T18:36:02.765280+0000","last_undegraded":"2024-04-23T18:36:02.765280+0000","last_fullsized":"2024-04-23T18:36:02.765280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:49.619906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764813+0000","last_change":"2024-04-23T18:34:01.225518+0000","last_active":"2024-04-23T18:36:02.764813+0000","last_peered":"2024-04-23T18:36:02.764813+0000","last_clean":"2024-04-23T18:36:02.764813+0000","last_became_active":"2024-04-23T18:34:01.224907+0000","last_became_peered":"2024-04-23T18:34:01.224907+0000","last_unstale":"2024-04-23T18:36:02.764813+0000","last_undegraded":"2024-04-23T18:36:02.764813+0000","last_fullsized":"2024-04-23T18:36:02.764813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:19.876629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":307,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.690858+0000","last_change":"2024-04-23T18:34:00.224770+0000","last_active":"2024-04-23T18:36:03.690858+0000","last_peered":"2024-04-23T18:36:03.690858+0000","last_clean":"2024-04-23T18:36:03.690858+0000","last_became_active":"2024-04-23T18:34:00.224395+0000","last_became_peered":"2024-04-23T18:34:00.224395+0000","last_unstale":"2024-04-23T18:36:03.690858+0000","last_undegraded":"2024-04-23T18:36:03.690858+0000","last_fullsized":"2024-04-23T18:36:03.690858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:14:09.172479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60022,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":248,"num_read_kb":183,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765184+0000","last_change":"2024-04-23T18:34:01.227771+0000","last_active":"2024-04-23T18:36:02.765184+0000","last_peered":"2024-04-23T18:36:02.765184+0000","last_clean":"2024-04-23T18:36:02.765184+0000","last_became_active":"2024-04-23T18:34:01.225063+0000","last_became_peered":"2024-04-23T18:34:01.225063+0000","last_unstale":"2024-04-23T18:36:02.765184+0000","last_undegraded":"2024-04-23T18:36:02.765184+0000","last_fullsized":"2024-04-23T18:36:02.765184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:53.921587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722785+0000","last_change":"2024-04-23T18:34:00.230455+0000","last_active":"2024-04-23T18:36:02.722785+0000","last_peered":"2024-04-23T18:36:02.722785+0000","last_clean":"2024-04-23T18:36:02.722785+0000","last_became_active":"2024-04-23T18:34:00.229934+0000","last_became_peered":"2024-04-23T18:34:00.229934+0000","last_unstale":"2024-04-23T18:36:02.722785+0000","last_undegraded":"2024-04-23T18:36:02.722785+0000","last_fullsized":"2024-04-23T18:36:02.722785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:26:34.787606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731340+0000","last_change":"2024-04-23T18:34:01.225757+0000","last_active":"2024-04-23T18:36:02.731340+0000","last_peered":"2024-04-23T18:36:02.731340+0000","last_clean":"2024-04-23T18:36:02.731340+0000","last_became_active":"2024-04-23T18:34:01.225447+0000","last_became_peered":"2024-04-23T18:34:01.225447+0000","last_unstale":"2024-04-23T18:36:02.731340+0000","last_undegraded":"2024-04-23T18:36:02.731340+0000","last_fullsized":"2024-04-23T18:36:02.731340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:17.212135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764698+0000","last_change":"2024-04-23T18:34:00.231749+0000","last_active":"2024-04-23T18:36:02.764698+0000","last_peered":"2024-04-23T18:36:02.764698+0000","last_clean":"2024-04-23T18:36:02.764698+0000","last_became_active":"2024-04-23T18:34:00.231207+0000","last_became_peered":"2024-04-23T18:34:00.231207+0000","last_unstale":"2024-04-23T18:36:02.764698+0000","last_undegraded":"2024-04-23T18:36:02.764698+0000","last_fullsized":"2024-04-23T18:36:02.764698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:23:05.723265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765316+0000","last_change":"2024-04-23T18:35:55.690568+0000","last_active":"2024-04-23T18:36:02.765316+0000","last_peered":"2024-04-23T18:36:02.765316+0000","last_clean":"2024-04-23T18:36:02.765316+0000","last_became_active":"2024-04-23T18:35:55.690174+0000","last_became_peered":"2024-04-23T18:35:55.690174+0000","last_unstale":"2024-04-23T18:36:02.765316+0000","last_undegraded":"2024-04-23T18:36:02.765316+0000","last_fullsized":"2024-04-23T18:36:02.765316+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:29.462962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.734130+0000","last_change":"2024-04-23T18:34:00.228578+0000","last_active":"2024-04-23T18:36:03.734130+0000","last_peered":"2024-04-23T18:36:03.734130+0000","last_clean":"2024-04-23T18:36:03.734130+0000","last_became_active":"2024-04-23T18:34:00.228238+0000","last_became_peered":"2024-04-23T18:34:00.228238+0000","last_unstale":"2024-04-23T18:36:03.734130+0000","last_undegraded":"2024-04-23T18:36:03.734130+0000","last_fullsized":"2024-04-23T18:36:03.734130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:58:21.283593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764889+0000","last_change":"2024-04-23T18:34:01.239481+0000","last_active":"2024-04-23T18:36:02.764889+0000","last_peered":"2024-04-23T18:36:02.764889+0000","last_clean":"2024-04-23T18:36:02.764889+0000","last_became_active":"2024-04-23T18:34:01.239042+0000","last_became_peered":"2024-04-23T18:34:01.239042+0000","last_unstale":"2024-04-23T18:36:02.764889+0000","last_undegraded":"2024-04-23T18:36:02.764889+0000","last_fullsized":"2024-04-23T18:36:02.764889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:45.755462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734712+0000","last_change":"2024-04-23T18:34:55.273529+0000","last_active":"2024-04-23T18:36:02.734712+0000","last_peered":"2024-04-23T18:36:02.734712+0000","last_clean":"2024-04-23T18:36:02.734712+0000","last_became_active":"2024-04-23T18:34:55.259780+0000","last_became_peered":"2024-04-23T18:34:55.259780+0000","last_unstale":"2024-04-23T18:36:02.734712+0000","last_undegraded":"2024-04-23T18:36:02.734712+0000","last_fullsized":"2024-04-23T18:36:02.734712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:48:56.272606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764515+0000","last_change":"2024-04-23T18:34:01.239943+0000","last_active":"2024-04-23T18:36:02.764515+0000","last_peered":"2024-04-23T18:36:02.764515+0000","last_clean":"2024-04-23T18:36:02.764515+0000","last_became_active":"2024-04-23T18:34:01.239588+0000","last_became_peered":"2024-04-23T18:34:01.239588+0000","last_unstale":"2024-04-23T18:36:02.764515+0000","last_undegraded":"2024-04-23T18:36:02.764515+0000","last_fullsized":"2024-04-23T18:36:02.764515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:51:39.344341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765578+0000","last_change":"2024-04-23T18:34:00.227372+0000","last_active":"2024-04-23T18:36:02.765578+0000","last_peered":"2024-04-23T18:36:02.765578+0000","last_clean":"2024-04-23T18:36:02.765578+0000","last_became_active":"2024-04-23T18:34:00.226709+0000","last_became_peered":"2024-04-23T18:34:00.226709+0000","last_unstale":"2024-04-23T18:36:02.765578+0000","last_undegraded":"2024-04-23T18:36:02.765578+0000","last_fullsized":"2024-04-23T18:36:02.765578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:20.047108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734890+0000","last_change":"2024-04-23T18:34:01.235389+0000","last_active":"2024-04-23T18:36:02.734890+0000","last_peered":"2024-04-23T18:36:02.734890+0000","last_clean":"2024-04-23T18:36:02.734890+0000","last_became_active":"2024-04-23T18:34:01.235026+0000","last_became_peered":"2024-04-23T18:34:01.235026+0000","last_unstale":"2024-04-23T18:36:02.734890+0000","last_undegraded":"2024-04-23T18:36:02.734890+0000","last_fullsized":"2024-04-23T18:36:02.734890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:48.849525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.721668+0000","last_change":"2024-04-23T18:34:55.260852+0000","last_active":"2024-04-23T18:36:02.721668+0000","last_peered":"2024-04-23T18:36:02.721668+0000","last_clean":"2024-04-23T18:36:02.721668+0000","last_became_active":"2024-04-23T18:34:55.260452+0000","last_became_peered":"2024-04-23T18:34:55.260452+0000","last_unstale":"2024-04-23T18:36:02.721668+0000","last_undegraded":"2024-04-23T18:36:02.721668+0000","last_fullsized":"2024-04-23T18:36:02.721668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:28:13.496447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732819+0000","last_change":"2024-04-23T18:34:01.224992+0000","last_active":"2024-04-23T18:36:02.732819+0000","last_peered":"2024-04-23T18:36:02.732819+0000","last_clean":"2024-04-23T18:36:02.732819+0000","last_became_active":"2024-04-23T18:34:01.224508+0000","last_became_peered":"2024-04-23T18:34:01.224508+0000","last_unstale":"2024-04-23T18:36:02.732819+0000","last_undegraded":"2024-04-23T18:36:02.732819+0000","last_fullsized":"2024-04-23T18:36:02.732819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:24.183451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":152,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.700066+0000","last_change":"2024-04-23T18:34:00.234818+0000","last_active":"2024-04-23T18:36:03.700066+0000","last_peered":"2024-04-23T18:36:03.700066+0000","last_clean":"2024-04-23T18:36:03.700066+0000","last_became_active":"2024-04-23T18:34:00.234329+0000","last_became_peered":"2024-04-23T18:34:00.234329+0000","last_unstale":"2024-04-23T18:36:03.700066+0000","last_undegraded":"2024-04-23T18:36:03.700066+0000","last_fullsized":"2024-04-23T18:36:03.700066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:43:02.405694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732868+0000","last_change":"2024-04-23T18:34:55.261073+0000","last_active":"2024-04-23T18:36:02.732868+0000","last_peered":"2024-04-23T18:36:02.732868+0000","last_clean":"2024-04-23T18:36:02.732868+0000","last_became_active":"2024-04-23T18:34:55.260655+0000","last_became_peered":"2024-04-23T18:34:55.260655+0000","last_unstale":"2024-04-23T18:36:02.732868+0000","last_undegraded":"2024-04-23T18:36:02.732868+0000","last_fullsized":"2024-04-23T18:36:02.732868+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:37:36.797785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764638+0000","last_change":"2024-04-23T18:34:01.234184+0000","last_active":"2024-04-23T18:36:02.764638+0000","last_peered":"2024-04-23T18:36:02.764638+0000","last_clean":"2024-04-23T18:36:02.764638+0000","last_became_active":"2024-04-23T18:34:01.233846+0000","last_became_peered":"2024-04-23T18:34:01.233846+0000","last_unstale":"2024-04-23T18:36:02.764638+0000","last_undegraded":"2024-04-23T18:36:02.764638+0000","last_fullsized":"2024-04-23T18:36:02.764638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:41.771495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.960951+0000","last_change":"2024-04-23T18:34:00.227862+0000","last_active":"2024-04-23T18:36:02.960951+0000","last_peered":"2024-04-23T18:36:02.960951+0000","last_clean":"2024-04-23T18:36:02.960951+0000","last_became_active":"2024-04-23T18:34:00.227515+0000","last_became_peered":"2024-04-23T18:34:00.227515+0000","last_unstale":"2024-04-23T18:36:02.960951+0000","last_undegraded":"2024-04-23T18:36:02.960951+0000","last_fullsized":"2024-04-23T18:36:02.960951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:18.595690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765280+0000","last_change":"2024-04-23T18:34:01.236338+0000","last_active":"2024-04-23T18:36:02.765280+0000","last_peered":"2024-04-23T18:36:02.765280+0000","last_clean":"2024-04-23T18:36:02.765280+0000","last_became_active":"2024-04-23T18:34:01.235795+0000","last_became_peered":"2024-04-23T18:34:01.235795+0000","last_unstale":"2024-04-23T18:36:02.765280+0000","last_undegraded":"2024-04-23T18:36:02.765280+0000","last_fullsized":"2024-04-23T18:36:02.765280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:45:01.928793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.725362+0000","last_change":"2024-04-23T18:34:00.221851+0000","last_active":"2024-04-23T18:36:02.725362+0000","last_peered":"2024-04-23T18:36:02.725362+0000","last_clean":"2024-04-23T18:36:02.725362+0000","last_became_active":"2024-04-23T18:34:00.221513+0000","last_became_peered":"2024-04-23T18:34:00.221513+0000","last_unstale":"2024-04-23T18:36:02.725362+0000","last_undegraded":"2024-04-23T18:36:02.725362+0000","last_fullsized":"2024-04-23T18:36:02.725362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:15:04.608759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764979+0000","last_change":"2024-04-23T18:32:48.218931+0000","last_active":"2024-04-23T18:36:02.764979+0000","last_peered":"2024-04-23T18:36:02.764979+0000","last_clean":"2024-04-23T18:36:02.764979+0000","last_became_active":"2024-04-23T18:32:48.218484+0000","last_became_peered":"2024-04-23T18:32:48.218484+0000","last_unstale":"2024-04-23T18:36:02.764979+0000","last_undegraded":"2024-04-23T18:36:02.764979+0000","last_fullsized":"2024-04-23T18:36:02.764979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_clean_scrub_stamp":"2024-04-23T18:29:21.029319+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:03:42.234165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731244+0000","last_change":"2024-04-23T18:34:01.220346+0000","last_active":"2024-04-23T18:36:02.731244+0000","last_peered":"2024-04-23T18:36:02.731244+0000","last_clean":"2024-04-23T18:36:02.731244+0000","last_became_active":"2024-04-23T18:34:01.219991+0000","last_became_peered":"2024-04-23T18:34:01.219991+0000","last_unstale":"2024-04-23T18:36:02.731244+0000","last_undegraded":"2024-04-23T18:36:02.731244+0000","last_fullsized":"2024-04-23T18:36:02.731244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:29:00.662700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765123+0000","last_change":"2024-04-23T18:34:00.229625+0000","last_active":"2024-04-23T18:36:02.765123+0000","last_peered":"2024-04-23T18:36:02.765123+0000","last_clean":"2024-04-23T18:36:02.765123+0000","last_became_active":"2024-04-23T18:34:00.229204+0000","last_became_peered":"2024-04-23T18:34:00.229204+0000","last_unstale":"2024-04-23T18:36:02.765123+0000","last_undegraded":"2024-04-23T18:36:02.765123+0000","last_fullsized":"2024-04-23T18:36:02.765123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:42.370870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765328+0000","last_change":"2024-04-23T18:34:00.236000+0000","last_active":"2024-04-23T18:36:02.765328+0000","last_peered":"2024-04-23T18:36:02.765328+0000","last_clean":"2024-04-23T18:36:02.765328+0000","last_became_active":"2024-04-23T18:34:00.235561+0000","last_became_peered":"2024-04-23T18:34:00.235561+0000","last_unstale":"2024-04-23T18:36:02.765328+0000","last_undegraded":"2024-04-23T18:36:02.765328+0000","last_fullsized":"2024-04-23T18:36:02.765328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:07.565489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731441+0000","last_change":"2024-04-23T18:34:01.227955+0000","last_active":"2024-04-23T18:36:02.731441+0000","last_peered":"2024-04-23T18:36:02.731441+0000","last_clean":"2024-04-23T18:36:02.731441+0000","last_became_active":"2024-04-23T18:34:01.225992+0000","last_became_peered":"2024-04-23T18:34:01.225992+0000","last_unstale":"2024-04-23T18:36:02.731441+0000","last_undegraded":"2024-04-23T18:36:02.731441+0000","last_fullsized":"2024-04-23T18:36:02.731441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:05:55.147008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'48","reported_seq":96,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.925135+0000","last_change":"2024-04-23T18:34:00.230133+0000","last_active":"2024-04-23T18:36:02.925135+0000","last_peered":"2024-04-23T18:36:02.925135+0000","last_clean":"2024-04-23T18:36:02.925135+0000","last_became_active":"2024-04-23T18:34:00.229596+0000","last_became_peered":"2024-04-23T18:34:00.229596+0000","last_unstale":"2024-04-23T18:36:02.925135+0000","last_undegraded":"2024-04-23T18:36:02.925135+0000","last_fullsized":"2024-04-23T18:36:02.925135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:23:06.064313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765529+0000","last_change":"2024-04-23T18:34:01.229066+0000","last_active":"2024-04-23T18:36:02.765529+0000","last_peered":"2024-04-23T18:36:02.765529+0000","last_clean":"2024-04-23T18:36:02.765529+0000","last_became_active":"2024-04-23T18:34:01.228711+0000","last_became_peered":"2024-04-23T18:34:01.228711+0000","last_unstale":"2024-04-23T18:36:02.765529+0000","last_undegraded":"2024-04-23T18:36:02.765529+0000","last_fullsized":"2024-04-23T18:36:02.765529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:58:30.601270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.725508+0000","last_change":"2024-04-23T18:34:00.231227+0000","last_active":"2024-04-23T18:36:02.725508+0000","last_peered":"2024-04-23T18:36:02.725508+0000","last_clean":"2024-04-23T18:36:02.725508+0000","last_became_active":"2024-04-23T18:34:00.230843+0000","last_became_peered":"2024-04-23T18:34:00.230843+0000","last_unstale":"2024-04-23T18:36:02.725508+0000","last_undegraded":"2024-04-23T18:36:02.725508+0000","last_fullsized":"2024-04-23T18:36:02.725508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:55.416591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765173+0000","last_change":"2024-04-23T18:34:01.233164+0000","last_active":"2024-04-23T18:36:02.765173+0000","last_peered":"2024-04-23T18:36:02.765173+0000","last_clean":"2024-04-23T18:36:02.765173+0000","last_became_active":"2024-04-23T18:34:01.232827+0000","last_became_peered":"2024-04-23T18:34:01.232827+0000","last_unstale":"2024-04-23T18:36:02.765173+0000","last_undegraded":"2024-04-23T18:36:02.765173+0000","last_fullsized":"2024-04-23T18:36:02.765173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:09.635515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764498+0000","last_change":"2024-04-23T18:34:00.225651+0000","last_active":"2024-04-23T18:36:02.764498+0000","last_peered":"2024-04-23T18:36:02.764498+0000","last_clean":"2024-04-23T18:36:02.764498+0000","last_became_active":"2024-04-23T18:34:00.225001+0000","last_became_peered":"2024-04-23T18:34:00.225001+0000","last_unstale":"2024-04-23T18:36:02.764498+0000","last_undegraded":"2024-04-23T18:36:02.764498+0000","last_fullsized":"2024-04-23T18:36:02.764498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:27:44.265388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765100+0000","last_change":"2024-04-23T18:34:01.224131+0000","last_active":"2024-04-23T18:36:02.765100+0000","last_peered":"2024-04-23T18:36:02.765100+0000","last_clean":"2024-04-23T18:36:02.765100+0000","last_became_active":"2024-04-23T18:34:01.223742+0000","last_became_peered":"2024-04-23T18:34:01.223742+0000","last_unstale":"2024-04-23T18:36:02.765100+0000","last_undegraded":"2024-04-23T18:36:02.765100+0000","last_fullsized":"2024-04-23T18:36:02.765100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:29:26.430964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764840+0000","last_change":"2024-04-23T18:34:00.235789+0000","last_active":"2024-04-23T18:36:02.764840+0000","last_peered":"2024-04-23T18:36:02.764840+0000","last_clean":"2024-04-23T18:36:02.764840+0000","last_became_active":"2024-04-23T18:34:00.235452+0000","last_became_peered":"2024-04-23T18:34:00.235452+0000","last_unstale":"2024-04-23T18:36:02.764840+0000","last_undegraded":"2024-04-23T18:36:02.764840+0000","last_fullsized":"2024-04-23T18:36:02.764840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:21.090121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734787+0000","last_change":"2024-04-23T18:34:01.226156+0000","last_active":"2024-04-23T18:36:02.734787+0000","last_peered":"2024-04-23T18:36:02.734787+0000","last_clean":"2024-04-23T18:36:02.734787+0000","last_became_active":"2024-04-23T18:34:01.225817+0000","last_became_peered":"2024-04-23T18:34:01.225817+0000","last_unstale":"2024-04-23T18:36:02.734787+0000","last_undegraded":"2024-04-23T18:36:02.734787+0000","last_fullsized":"2024-04-23T18:36:02.734787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:36.337139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.723101+0000","last_change":"2024-04-23T18:34:00.235116+0000","last_active":"2024-04-23T18:36:02.723101+0000","last_peered":"2024-04-23T18:36:02.723101+0000","last_clean":"2024-04-23T18:36:02.723101+0000","last_became_active":"2024-04-23T18:34:00.234521+0000","last_became_peered":"2024-04-23T18:34:00.234521+0000","last_unstale":"2024-04-23T18:36:02.723101+0000","last_undegraded":"2024-04-23T18:36:02.723101+0000","last_fullsized":"2024-04-23T18:36:02.723101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:51.061164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765111+0000","last_change":"2024-04-23T18:34:01.226445+0000","last_active":"2024-04-23T18:36:02.765111+0000","last_peered":"2024-04-23T18:36:02.765111+0000","last_clean":"2024-04-23T18:36:02.765111+0000","last_became_active":"2024-04-23T18:34:01.225875+0000","last_became_peered":"2024-04-23T18:34:01.225875+0000","last_unstale":"2024-04-23T18:36:02.765111+0000","last_undegraded":"2024-04-23T18:36:02.765111+0000","last_fullsized":"2024-04-23T18:36:02.765111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:48:38.712502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765246+0000","last_change":"2024-04-23T18:34:00.231918+0000","last_active":"2024-04-23T18:36:02.765246+0000","last_peered":"2024-04-23T18:36:02.765246+0000","last_clean":"2024-04-23T18:36:02.765246+0000","last_became_active":"2024-04-23T18:34:00.231550+0000","last_became_peered":"2024-04-23T18:34:00.231550+0000","last_unstale":"2024-04-23T18:36:02.765246+0000","last_undegraded":"2024-04-23T18:36:02.765246+0000","last_fullsized":"2024-04-23T18:36:02.765246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:52.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765279+0000","last_change":"2024-04-23T18:34:01.236360+0000","last_active":"2024-04-23T18:36:02.765279+0000","last_peered":"2024-04-23T18:36:02.765279+0000","last_clean":"2024-04-23T18:36:02.765279+0000","last_became_active":"2024-04-23T18:34:01.236056+0000","last_became_peered":"2024-04-23T18:34:01.236056+0000","last_unstale":"2024-04-23T18:36:02.765279+0000","last_undegraded":"2024-04-23T18:36:02.765279+0000","last_fullsized":"2024-04-23T18:36:02.765279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:54.535158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'9","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.736528+0000","last_change":"2024-04-23T18:34:00.233159+0000","last_active":"2024-04-23T18:36:03.736528+0000","last_peered":"2024-04-23T18:36:03.736528+0000","last_clean":"2024-04-23T18:36:03.736528+0000","last_became_active":"2024-04-23T18:34:00.232728+0000","last_became_peered":"2024-04-23T18:34:00.232728+0000","last_unstale":"2024-04-23T18:36:03.736528+0000","last_undegraded":"2024-04-23T18:36:03.736528+0000","last_fullsized":"2024-04-23T18:36:03.736528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:23.498057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2484,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765647+0000","last_change":"2024-04-23T18:34:01.226134+0000","last_active":"2024-04-23T18:36:02.765647+0000","last_peered":"2024-04-23T18:36:02.765647+0000","last_clean":"2024-04-23T18:36:02.765647+0000","last_became_active":"2024-04-23T18:34:01.225759+0000","last_became_peered":"2024-04-23T18:34:01.225759+0000","last_unstale":"2024-04-23T18:36:02.765647+0000","last_undegraded":"2024-04-23T18:36:02.765647+0000","last_fullsized":"2024-04-23T18:36:02.765647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:24.770595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.923105+0000","last_change":"2024-04-23T18:34:00.230054+0000","last_active":"2024-04-23T18:36:02.923105+0000","last_peered":"2024-04-23T18:36:02.923105+0000","last_clean":"2024-04-23T18:36:02.923105+0000","last_became_active":"2024-04-23T18:34:00.229621+0000","last_became_peered":"2024-04-23T18:34:00.229621+0000","last_unstale":"2024-04-23T18:36:02.923105+0000","last_undegraded":"2024-04-23T18:36:02.923105+0000","last_fullsized":"2024-04-23T18:36:02.923105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:13:59.317282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765433+0000","last_change":"2024-04-23T18:34:01.240053+0000","last_active":"2024-04-23T18:36:02.765433+0000","last_peered":"2024-04-23T18:36:02.765433+0000","last_clean":"2024-04-23T18:36:02.765433+0000","last_became_active":"2024-04-23T18:34:01.239716+0000","last_became_peered":"2024-04-23T18:34:01.239716+0000","last_unstale":"2024-04-23T18:36:02.765433+0000","last_undegraded":"2024-04-23T18:36:02.765433+0000","last_fullsized":"2024-04-23T18:36:02.765433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:54.874374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731781+0000","last_change":"2024-04-23T18:34:00.227914+0000","last_active":"2024-04-23T18:36:02.731781+0000","last_peered":"2024-04-23T18:36:02.731781+0000","last_clean":"2024-04-23T18:36:02.731781+0000","last_became_active":"2024-04-23T18:34:00.227540+0000","last_became_peered":"2024-04-23T18:34:00.227540+0000","last_unstale":"2024-04-23T18:36:02.731781+0000","last_undegraded":"2024-04-23T18:36:02.731781+0000","last_fullsized":"2024-04-23T18:36:02.731781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:18.442397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765656+0000","last_change":"2024-04-23T18:34:01.227319+0000","last_active":"2024-04-23T18:36:02.765656+0000","last_peered":"2024-04-23T18:36:02.765656+0000","last_clean":"2024-04-23T18:36:02.765656+0000","last_became_active":"2024-04-23T18:34:01.226969+0000","last_became_peered":"2024-04-23T18:34:01.226969+0000","last_unstale":"2024-04-23T18:36:02.765656+0000","last_undegraded":"2024-04-23T18:36:02.765656+0000","last_fullsized":"2024-04-23T18:36:02.765656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:19.302388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734899+0000","last_change":"2024-04-23T18:34:01.234466+0000","last_active":"2024-04-23T18:36:02.734899+0000","last_peered":"2024-04-23T18:36:02.734899+0000","last_clean":"2024-04-23T18:36:02.734899+0000","last_became_active":"2024-04-23T18:34:01.234127+0000","last_became_peered":"2024-04-23T18:34:01.234127+0000","last_unstale":"2024-04-23T18:36:02.734899+0000","last_undegraded":"2024-04-23T18:36:02.734899+0000","last_fullsized":"2024-04-23T18:36:02.734899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:51:51.912320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722721+0000","last_change":"2024-04-23T18:34:01.229707+0000","last_active":"2024-04-23T18:36:02.722721+0000","last_peered":"2024-04-23T18:36:02.722721+0000","last_clean":"2024-04-23T18:36:02.722721+0000","last_became_active":"2024-04-23T18:34:01.229311+0000","last_became_peered":"2024-04-23T18:34:01.229311+0000","last_unstale":"2024-04-23T18:36:02.722721+0000","last_undegraded":"2024-04-23T18:36:02.722721+0000","last_fullsized":"2024-04-23T18:36:02.722721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:15:51.972436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722618+0000","last_change":"2024-04-23T18:34:01.227285+0000","last_active":"2024-04-23T18:36:02.722618+0000","last_peered":"2024-04-23T18:36:02.722618+0000","last_clean":"2024-04-23T18:36:02.722618+0000","last_became_active":"2024-04-23T18:34:01.226960+0000","last_became_peered":"2024-04-23T18:34:01.226960+0000","last_unstale":"2024-04-23T18:36:02.722618+0000","last_undegraded":"2024-04-23T18:36:02.722618+0000","last_fullsized":"2024-04-23T18:36:02.722618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:50.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764963+0000","last_change":"2024-04-23T18:34:01.230697+0000","last_active":"2024-04-23T18:36:02.764963+0000","last_peered":"2024-04-23T18:36:02.764963+0000","last_clean":"2024-04-23T18:36:02.764963+0000","last_became_active":"2024-04-23T18:34:01.230352+0000","last_became_peered":"2024-04-23T18:34:01.230352+0000","last_unstale":"2024-04-23T18:36:02.764963+0000","last_undegraded":"2024-04-23T18:36:02.764963+0000","last_fullsized":"2024-04-23T18:36:02.764963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:35:10.609773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731598+0000","last_change":"2024-04-23T18:34:01.226970+0000","last_active":"2024-04-23T18:36:02.731598+0000","last_peered":"2024-04-23T18:36:02.731598+0000","last_clean":"2024-04-23T18:36:02.731598+0000","last_became_active":"2024-04-23T18:34:01.224151+0000","last_became_peered":"2024-04-23T18:34:01.224151+0000","last_unstale":"2024-04-23T18:36:02.731598+0000","last_undegraded":"2024-04-23T18:36:02.731598+0000","last_fullsized":"2024-04-23T18:36:02.731598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:36:15.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734930+0000","last_change":"2024-04-23T18:34:01.234194+0000","last_active":"2024-04-23T18:36:02.734930+0000","last_peered":"2024-04-23T18:36:02.734930+0000","last_clean":"2024-04-23T18:36:02.734930+0000","last_became_active":"2024-04-23T18:34:01.232404+0000","last_became_peered":"2024-04-23T18:34:01.232404+0000","last_unstale":"2024-04-23T18:36:02.734930+0000","last_undegraded":"2024-04-23T18:36:02.734930+0000","last_fullsized":"2024-04-23T18:36:02.734930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:30.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722974+0000","last_change":"2024-04-23T18:34:01.219507+0000","last_active":"2024-04-23T18:36:02.722974+0000","last_peered":"2024-04-23T18:36:02.722974+0000","last_clean":"2024-04-23T18:36:02.722974+0000","last_became_active":"2024-04-23T18:34:01.218997+0000","last_became_peered":"2024-04-23T18:34:01.218997+0000","last_unstale":"2024-04-23T18:36:02.722974+0000","last_undegraded":"2024-04-23T18:36:02.722974+0000","last_fullsized":"2024-04-23T18:36:02.722974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:38.233652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765439+0000","last_change":"2024-04-23T18:34:01.231846+0000","last_active":"2024-04-23T18:36:02.765439+0000","last_peered":"2024-04-23T18:36:02.765439+0000","last_clean":"2024-04-23T18:36:02.765439+0000","last_became_active":"2024-04-23T18:34:01.231467+0000","last_became_peered":"2024-04-23T18:34:01.231467+0000","last_unstale":"2024-04-23T18:36:02.765439+0000","last_undegraded":"2024-04-23T18:36:02.765439+0000","last_fullsized":"2024-04-23T18:36:02.765439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:37:27.891180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764528+0000","last_change":"2024-04-23T18:34:01.237086+0000","last_active":"2024-04-23T18:36:02.764528+0000","last_peered":"2024-04-23T18:36:02.764528+0000","last_clean":"2024-04-23T18:36:02.764528+0000","last_became_active":"2024-04-23T18:34:01.236610+0000","last_became_peered":"2024-04-23T18:34:01.236610+0000","last_unstale":"2024-04-23T18:36:02.764528+0000","last_undegraded":"2024-04-23T18:36:02.764528+0000","last_fullsized":"2024-04-23T18:36:02.764528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:24.916553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764832+0000","last_change":"2024-04-23T18:34:01.231091+0000","last_active":"2024-04-23T18:36:02.764832+0000","last_peered":"2024-04-23T18:36:02.764832+0000","last_clean":"2024-04-23T18:36:02.764832+0000","last_became_active":"2024-04-23T18:34:01.230731+0000","last_became_peered":"2024-04-23T18:34:01.230731+0000","last_unstale":"2024-04-23T18:36:02.764832+0000","last_undegraded":"2024-04-23T18:36:02.764832+0000","last_fullsized":"2024-04-23T18:36:02.764832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:22.771515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764631+0000","last_change":"2024-04-23T18:34:01.231820+0000","last_active":"2024-04-23T18:36:02.764631+0000","last_peered":"2024-04-23T18:36:02.764631+0000","last_clean":"2024-04-23T18:36:02.764631+0000","last_became_active":"2024-04-23T18:34:01.230320+0000","last_became_peered":"2024-04-23T18:34:01.230320+0000","last_unstale":"2024-04-23T18:36:02.764631+0000","last_undegraded":"2024-04-23T18:36:02.764631+0000","last_fullsized":"2024-04-23T18:36:02.764631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:27:35.324323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731327+0000","last_change":"2024-04-23T18:34:01.228723+0000","last_active":"2024-04-23T18:36:02.731327+0000","last_peered":"2024-04-23T18:36:02.731327+0000","last_clean":"2024-04-23T18:36:02.731327+0000","last_became_active":"2024-04-23T18:34:01.227500+0000","last_became_peered":"2024-04-23T18:34:01.227500+0000","last_unstale":"2024-04-23T18:36:02.731327+0000","last_undegraded":"2024-04-23T18:36:02.731327+0000","last_fullsized":"2024-04-23T18:36:02.731327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:17.908021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734867+0000","last_change":"2024-04-23T18:34:01.227541+0000","last_active":"2024-04-23T18:36:02.734867+0000","last_peered":"2024-04-23T18:36:02.734867+0000","last_clean":"2024-04-23T18:36:02.734867+0000","last_became_active":"2024-04-23T18:34:01.227201+0000","last_became_peered":"2024-04-23T18:34:01.227201+0000","last_unstale":"2024-04-23T18:36:02.734867+0000","last_undegraded":"2024-04-23T18:36:02.734867+0000","last_fullsized":"2024-04-23T18:36:02.734867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:25:24.886850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107658,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":594196,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1319,"num_read_kb":769,"num_write":371,"num_write_kb":1459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":688128,"data_stored":1782588,"data_compressed":81528,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":488912,"data_compressed":14490,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1171456,"data_stored":1271310,"data_compressed":58406,"data_compressed_allocated":77824,"data_compressed_original":761856,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":881414,"data_compressed":33636,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037813,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":881680,"data_compressed":33636,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":1081543,"data_compressed":50376,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495680,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":488614,"data_compressed":14490,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":549158,"data_compressed":16364,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":489786,"data_compressed":14490,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":550134,"data_compressed":16364,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1012556,"data_compressed":39792,"data_compressed_allocated":53248,"data_compressed_original":557056,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":488167,"data_compressed":14490,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1080913,"data_compressed":50376,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62473,"data_compressed":1874,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63840,"data_compressed":1874,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134952,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":193749,"data_compressed":8030,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:36:08.292 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:36:08.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.433+0000 7fb368b7d640 1 -- 172.21.15.132:0/3178206900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb364103230 msgr2=0x7fb364103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.433+0000 7fb368b7d640 1 --2- 172.21.15.132:0/3178206900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb364103230 0x7fb364103610 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb354009900 tx=0x7fb35402a480 comp rx=0 tx=0).stop 2024-04-23T18:36:08.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.434+0000 7fb368b7d640 1 -- 172.21.15.132:0/3178206900 shutdown_connections 2024-04-23T18:36:08.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.434+0000 7fb368b7d640 1 --2- 172.21.15.132:0/3178206900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb36410c570 0x7fb36410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.434+0000 7fb368b7d640 1 --2- 172.21.15.132:0/3178206900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb364103b50 0x7fb36410c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.434+0000 7fb368b7d640 1 --2- 172.21.15.132:0/3178206900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb364103230 0x7fb364103610 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.434+0000 7fb368b7d640 1 -- 172.21.15.132:0/3178206900 >> 172.21.15.132:0/3178206900 conn(0x7fb364077fb0 msgr2=0x7fb3640783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:08.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.434+0000 7fb368b7d640 1 -- 172.21.15.132:0/3178206900 shutdown_connections 2024-04-23T18:36:08.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.434+0000 7fb368b7d640 1 -- 172.21.15.132:0/3178206900 wait complete. 2024-04-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.434+0000 7fb368b7d640 1 Processor -- start 2024-04-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.434+0000 7fb368b7d640 1 -- start start 2024-04-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb368b7d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb364103230 0x7fb364100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb368b7d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb364103b50 0x7fb3641013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb368b7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb36410c570 0x7fb364101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb368b7d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb3641121d0 con 0x7fb364103230 2024-04-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb368b7d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb364112050 con 0x7fb364103b50 2024-04-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb368b7d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb364112350 con 0x7fb36410c570 2024-04-23T18:36:08.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb36259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb364103230 0x7fb364100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb36259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb364103230 0x7fb364100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57824/0 (socket says 172.21.15.132:57824) 2024-04-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb36259c640 1 -- 172.21.15.132:0/2932000680 learned_addr learned my addr 172.21.15.132:0/2932000680 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb361d9b640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb364103b50 0x7fb3641013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb362d9d640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb36410c570 0x7fb364101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb36259c640 1 -- 172.21.15.132:0/2932000680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb36410c570 msgr2=0x7fb364101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb36259c640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb36410c570 0x7fb364101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb36259c640 1 -- 172.21.15.132:0/2932000680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb364103b50 msgr2=0x7fb3641013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb36259c640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb364103b50 0x7fb3641013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb36259c640 1 -- 172.21.15.132:0/2932000680 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3641a6bd0 con 0x7fb364103230 2024-04-23T18:36:08.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb361d9b640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb364103b50 0x7fb3641013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:08.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb36259c640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb364103230 0x7fb364100e80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb354005940 tx=0x7fb3540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:08.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb3537fe640 1 -- 172.21.15.132:0/2932000680 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35403f030 con 0x7fb364103230 2024-04-23T18:36:08.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb3537fe640 1 -- 172.21.15.132:0/2932000680 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb354003cb0 con 0x7fb364103230 2024-04-23T18:36:08.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb368b7d640 1 -- 172.21.15.132:0/2932000680 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3641a6e60 con 0x7fb364103230 2024-04-23T18:36:08.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb3537fe640 1 -- 172.21.15.132:0/2932000680 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354035070 con 0x7fb364103230 2024-04-23T18:36:08.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.435+0000 7fb368b7d640 1 -- 172.21.15.132:0/2932000680 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3641a72c0 con 0x7fb364103230 2024-04-23T18:36:08.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.436+0000 7fb368b7d640 1 -- 172.21.15.132:0/2932000680 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3640684e0 con 0x7fb364103230 2024-04-23T18:36:08.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.437+0000 7fb3537fe640 1 -- 172.21.15.132:0/2932000680 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fb354042030 con 0x7fb364103230 2024-04-23T18:36:08.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.438+0000 7fb3537fe640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fb34c0a3e40 0x7fb34c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:08.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.438+0000 7fb3537fe640 1 -- 172.21.15.132:0/2932000680 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fb3540f12f0 con 0x7fb364103230 2024-04-23T18:36:08.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.438+0000 7fb361d9b640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fb34c0a3e40 0x7fb34c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:08.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.438+0000 7fb361d9b640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fb34c0a3e40 0x7fb34c0a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb364102910 tx=0x7fb35800a5c0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:08.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.442+0000 7fb3537fe640 1 -- 172.21.15.132:0/2932000680 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3540bd800 con 0x7fb364103230 2024-04-23T18:36:08.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.617+0000 7fb368b7d640 1 -- 172.21.15.132:0/2932000680 --> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb364062390 con 0x7fb34c0a3e40 2024-04-23T18:36:08.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.624+0000 7fb3537fe640 1 -- 172.21.15.132:0/2932000680 <== mgr.15366 v2:172.21.15.132:6800/3428544997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258101 (secure 0 0 0) 0x7fb364062390 con 0x7fb34c0a3e40 2024-04-23T18:36:08.619 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-04-23T18:36:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.628+0000 7fb368b7d640 1 -- 172.21.15.132:0/2932000680 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fb34c0a3e40 msgr2=0x7fb34c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.628+0000 7fb368b7d640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fb34c0a3e40 0x7fb34c0a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb364102910 tx=0x7fb35800a5c0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.628+0000 7fb368b7d640 1 -- 172.21.15.132:0/2932000680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb364103230 msgr2=0x7fb364100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.628+0000 7fb368b7d640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb364103230 0x7fb364100e80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb354005940 tx=0x7fb3540042f0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.628+0000 7fb368b7d640 1 -- 172.21.15.132:0/2932000680 shutdown_connections 2024-04-23T18:36:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.628+0000 7fb368b7d640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fb34c0a3e40 0x7fb34c0a6300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.628+0000 7fb368b7d640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb36410c570 0x7fb364101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.628+0000 7fb368b7d640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb364103b50 0x7fb3641013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.628+0000 7fb368b7d640 1 --2- 172.21.15.132:0/2932000680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb364103230 0x7fb364100e80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.628+0000 7fb368b7d640 1 -- 172.21.15.132:0/2932000680 >> 172.21.15.132:0/2932000680 conn(0x7fb364077fb0 msgr2=0x7fb3640fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.628+0000 7fb368b7d640 1 -- 172.21.15.132:0/2932000680 shutdown_connections 2024-04-23T18:36:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.628+0000 7fb368b7d640 1 -- 172.21.15.132:0/2932000680 wait complete. 2024-04-23T18:36:08.631 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-23T18:36:06.735487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1184798,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1455,"num_read_kb":888,"num_write":554,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353260,"kb_used_data":12388,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588564,"statfs":{"total":1151940427776,"available":1151578689536,"internally_reserved":0,"allocated":12685312,"data_stored":9264187,"data_compressed":356910,"data_compressed_allocated":483328,"data_compressed_original":3710976,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990809"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734914+0000","last_change":"2024-04-23T18:34:01.231204+0000","last_active":"2024-04-23T18:36:02.734914+0000","last_peered":"2024-04-23T18:36:02.734914+0000","last_clean":"2024-04-23T18:36:02.734914+0000","last_became_active":"2024-04-23T18:34:01.230836+0000","last_became_peered":"2024-04-23T18:34:01.230836+0000","last_unstale":"2024-04-23T18:36:02.734914+0000","last_undegraded":"2024-04-23T18:36:02.734914+0000","last_fullsized":"2024-04-23T18:36:02.734914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:57.744807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765309+0000","last_change":"2024-04-23T18:34:01.225833+0000","last_active":"2024-04-23T18:36:02.765309+0000","last_peered":"2024-04-23T18:36:02.765309+0000","last_clean":"2024-04-23T18:36:02.765309+0000","last_became_active":"2024-04-23T18:34:01.225098+0000","last_became_peered":"2024-04-23T18:34:01.225098+0000","last_unstale":"2024-04-23T18:36:02.765309+0000","last_undegraded":"2024-04-23T18:36:02.765309+0000","last_fullsized":"2024-04-23T18:36:02.765309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:20.720137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765240+0000","last_change":"2024-04-23T18:34:01.235269+0000","last_active":"2024-04-23T18:36:02.765240+0000","last_peered":"2024-04-23T18:36:02.765240+0000","last_clean":"2024-04-23T18:36:02.765240+0000","last_became_active":"2024-04-23T18:34:01.234921+0000","last_became_peered":"2024-04-23T18:34:01.234921+0000","last_unstale":"2024-04-23T18:36:02.765240+0000","last_undegraded":"2024-04-23T18:36:02.765240+0000","last_fullsized":"2024-04-23T18:36:02.765240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:06:20.645693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.725703+0000","last_change":"2024-04-23T18:34:01.233316+0000","last_active":"2024-04-23T18:36:02.725703+0000","last_peered":"2024-04-23T18:36:02.725703+0000","last_clean":"2024-04-23T18:36:02.725703+0000","last_became_active":"2024-04-23T18:34:01.231636+0000","last_became_peered":"2024-04-23T18:34:01.231636+0000","last_unstale":"2024-04-23T18:36:02.725703+0000","last_undegraded":"2024-04-23T18:36:02.725703+0000","last_fullsized":"2024-04-23T18:36:02.725703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:15.837129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765118+0000","last_change":"2024-04-23T18:34:01.228073+0000","last_active":"2024-04-23T18:36:02.765118+0000","last_peered":"2024-04-23T18:36:02.765118+0000","last_clean":"2024-04-23T18:36:02.765118+0000","last_became_active":"2024-04-23T18:34:01.227701+0000","last_became_peered":"2024-04-23T18:34:01.227701+0000","last_unstale":"2024-04-23T18:36:02.765118+0000","last_undegraded":"2024-04-23T18:36:02.765118+0000","last_fullsized":"2024-04-23T18:36:02.765118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:17:26.556498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732650+0000","last_change":"2024-04-23T18:34:55.258233+0000","last_active":"2024-04-23T18:36:02.732650+0000","last_peered":"2024-04-23T18:36:02.732650+0000","last_clean":"2024-04-23T18:36:02.732650+0000","last_became_active":"2024-04-23T18:34:55.257836+0000","last_became_peered":"2024-04-23T18:34:55.257836+0000","last_unstale":"2024-04-23T18:36:02.732650+0000","last_undegraded":"2024-04-23T18:36:02.732650+0000","last_fullsized":"2024-04-23T18:36:02.732650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:20.279369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765125+0000","last_change":"2024-04-23T18:34:55.260737+0000","last_active":"2024-04-23T18:36:02.765125+0000","last_peered":"2024-04-23T18:36:02.765125+0000","last_clean":"2024-04-23T18:36:02.765125+0000","last_became_active":"2024-04-23T18:34:55.260231+0000","last_became_peered":"2024-04-23T18:34:55.260231+0000","last_unstale":"2024-04-23T18:36:02.765125+0000","last_undegraded":"2024-04-23T18:36:02.765125+0000","last_fullsized":"2024-04-23T18:36:02.765125+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:45.292690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.721402+0000","last_change":"2024-04-23T18:34:55.259963+0000","last_active":"2024-04-23T18:36:02.721402+0000","last_peered":"2024-04-23T18:36:02.721402+0000","last_clean":"2024-04-23T18:36:02.721402+0000","last_became_active":"2024-04-23T18:34:55.259342+0000","last_became_peered":"2024-04-23T18:34:55.259342+0000","last_unstale":"2024-04-23T18:36:02.721402+0000","last_undegraded":"2024-04-23T18:36:02.721402+0000","last_fullsized":"2024-04-23T18:36:02.721402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:42:36.612809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765111+0000","last_change":"2024-04-23T18:34:01.234094+0000","last_active":"2024-04-23T18:36:02.765111+0000","last_peered":"2024-04-23T18:36:02.765111+0000","last_clean":"2024-04-23T18:36:02.765111+0000","last_became_active":"2024-04-23T18:34:01.232499+0000","last_became_peered":"2024-04-23T18:34:01.232499+0000","last_unstale":"2024-04-23T18:36:02.765111+0000","last_undegraded":"2024-04-23T18:36:02.765111+0000","last_fullsized":"2024-04-23T18:36:02.765111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:33.004245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732991+0000","last_change":"2024-04-23T18:34:01.229175+0000","last_active":"2024-04-23T18:36:02.732991+0000","last_peered":"2024-04-23T18:36:02.732991+0000","last_clean":"2024-04-23T18:36:02.732991+0000","last_became_active":"2024-04-23T18:34:01.228837+0000","last_became_peered":"2024-04-23T18:34:01.228837+0000","last_unstale":"2024-04-23T18:36:02.732991+0000","last_undegraded":"2024-04-23T18:36:02.732991+0000","last_fullsized":"2024-04-23T18:36:02.732991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:57.625110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765546+0000","last_change":"2024-04-23T18:34:01.235557+0000","last_active":"2024-04-23T18:36:02.765546+0000","last_peered":"2024-04-23T18:36:02.765546+0000","last_clean":"2024-04-23T18:36:02.765546+0000","last_became_active":"2024-04-23T18:34:01.235184+0000","last_became_peered":"2024-04-23T18:34:01.235184+0000","last_unstale":"2024-04-23T18:36:02.765546+0000","last_undegraded":"2024-04-23T18:36:02.765546+0000","last_fullsized":"2024-04-23T18:36:02.765546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:37:27.138041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.725500+0000","last_change":"2024-04-23T18:34:01.234070+0000","last_active":"2024-04-23T18:36:02.725500+0000","last_peered":"2024-04-23T18:36:02.725500+0000","last_clean":"2024-04-23T18:36:02.725500+0000","last_became_active":"2024-04-23T18:34:01.233727+0000","last_became_peered":"2024-04-23T18:34:01.233727+0000","last_unstale":"2024-04-23T18:36:02.725500+0000","last_undegraded":"2024-04-23T18:36:02.725500+0000","last_fullsized":"2024-04-23T18:36:02.725500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:59:16.489819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731502+0000","last_change":"2024-04-23T18:34:01.228256+0000","last_active":"2024-04-23T18:36:02.731502+0000","last_peered":"2024-04-23T18:36:02.731502+0000","last_clean":"2024-04-23T18:36:02.731502+0000","last_became_active":"2024-04-23T18:34:01.224400+0000","last_became_peered":"2024-04-23T18:34:01.224400+0000","last_unstale":"2024-04-23T18:36:02.731502+0000","last_undegraded":"2024-04-23T18:36:02.731502+0000","last_fullsized":"2024-04-23T18:36:02.731502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:37:28.401695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764642+0000","last_change":"2024-04-23T18:34:01.227250+0000","last_active":"2024-04-23T18:36:02.764642+0000","last_peered":"2024-04-23T18:36:02.764642+0000","last_clean":"2024-04-23T18:36:02.764642+0000","last_became_active":"2024-04-23T18:34:01.226937+0000","last_became_peered":"2024-04-23T18:34:01.226937+0000","last_unstale":"2024-04-23T18:36:02.764642+0000","last_undegraded":"2024-04-23T18:36:02.764642+0000","last_fullsized":"2024-04-23T18:36:02.764642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:26:20.801691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764582+0000","last_change":"2024-04-23T18:34:01.232351+0000","last_active":"2024-04-23T18:36:02.764582+0000","last_peered":"2024-04-23T18:36:02.764582+0000","last_clean":"2024-04-23T18:36:02.764582+0000","last_became_active":"2024-04-23T18:34:01.231987+0000","last_became_peered":"2024-04-23T18:34:01.231987+0000","last_unstale":"2024-04-23T18:36:02.764582+0000","last_undegraded":"2024-04-23T18:36:02.764582+0000","last_fullsized":"2024-04-23T18:36:02.764582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:50.886496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764465+0000","last_change":"2024-04-23T18:34:01.232196+0000","last_active":"2024-04-23T18:36:02.764465+0000","last_peered":"2024-04-23T18:36:02.764465+0000","last_clean":"2024-04-23T18:36:02.764465+0000","last_became_active":"2024-04-23T18:34:01.231746+0000","last_became_peered":"2024-04-23T18:34:01.231746+0000","last_unstale":"2024-04-23T18:36:02.764465+0000","last_undegraded":"2024-04-23T18:36:02.764465+0000","last_fullsized":"2024-04-23T18:36:02.764465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:58.888549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.735096+0000","last_change":"2024-04-23T18:34:01.231509+0000","last_active":"2024-04-23T18:36:02.735096+0000","last_peered":"2024-04-23T18:36:02.735096+0000","last_clean":"2024-04-23T18:36:02.735096+0000","last_became_active":"2024-04-23T18:34:01.231017+0000","last_became_peered":"2024-04-23T18:34:01.231017+0000","last_unstale":"2024-04-23T18:36:02.735096+0000","last_undegraded":"2024-04-23T18:36:02.735096+0000","last_fullsized":"2024-04-23T18:36:02.735096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:04:51.929462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.721702+0000","last_change":"2024-04-23T18:34:01.223285+0000","last_active":"2024-04-23T18:36:02.721702+0000","last_peered":"2024-04-23T18:36:02.721702+0000","last_clean":"2024-04-23T18:36:02.721702+0000","last_became_active":"2024-04-23T18:34:01.222962+0000","last_became_peered":"2024-04-23T18:34:01.222962+0000","last_unstale":"2024-04-23T18:36:02.721702+0000","last_undegraded":"2024-04-23T18:36:02.721702+0000","last_fullsized":"2024-04-23T18:36:02.721702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:18.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722773+0000","last_change":"2024-04-23T18:34:01.225922+0000","last_active":"2024-04-23T18:36:02.722773+0000","last_peered":"2024-04-23T18:36:02.722773+0000","last_clean":"2024-04-23T18:36:02.722773+0000","last_became_active":"2024-04-23T18:34:01.225235+0000","last_became_peered":"2024-04-23T18:34:01.225235+0000","last_unstale":"2024-04-23T18:36:02.722773+0000","last_undegraded":"2024-04-23T18:36:02.722773+0000","last_fullsized":"2024-04-23T18:36:02.722773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:33.480782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.699597+0000","last_change":"2024-04-23T18:34:00.224606+0000","last_active":"2024-04-23T18:36:03.699597+0000","last_peered":"2024-04-23T18:36:03.699597+0000","last_clean":"2024-04-23T18:36:03.699597+0000","last_became_active":"2024-04-23T18:34:00.224291+0000","last_became_peered":"2024-04-23T18:34:00.224291+0000","last_unstale":"2024-04-23T18:36:03.699597+0000","last_undegraded":"2024-04-23T18:36:03.699597+0000","last_fullsized":"2024-04-23T18:36:03.699597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:40.734704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765428+0000","last_change":"2024-04-23T18:34:01.235023+0000","last_active":"2024-04-23T18:36:02.765428+0000","last_peered":"2024-04-23T18:36:02.765428+0000","last_clean":"2024-04-23T18:36:02.765428+0000","last_became_active":"2024-04-23T18:34:01.234672+0000","last_became_peered":"2024-04-23T18:34:01.234672+0000","last_unstale":"2024-04-23T18:36:02.765428+0000","last_undegraded":"2024-04-23T18:36:02.765428+0000","last_fullsized":"2024-04-23T18:36:02.765428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:33.734343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765629+0000","last_change":"2024-04-23T18:34:00.232552+0000","last_active":"2024-04-23T18:36:02.765629+0000","last_peered":"2024-04-23T18:36:02.765629+0000","last_clean":"2024-04-23T18:36:02.765629+0000","last_became_active":"2024-04-23T18:34:00.232226+0000","last_became_peered":"2024-04-23T18:34:00.232226+0000","last_unstale":"2024-04-23T18:36:02.765629+0000","last_undegraded":"2024-04-23T18:36:02.765629+0000","last_fullsized":"2024-04-23T18:36:02.765629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.563757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765732+0000","last_change":"2024-04-23T18:34:01.236013+0000","last_active":"2024-04-23T18:36:02.765732+0000","last_peered":"2024-04-23T18:36:02.765732+0000","last_clean":"2024-04-23T18:36:02.765732+0000","last_became_active":"2024-04-23T18:34:01.235590+0000","last_became_peered":"2024-04-23T18:34:01.235590+0000","last_unstale":"2024-04-23T18:36:02.765732+0000","last_undegraded":"2024-04-23T18:36:02.765732+0000","last_fullsized":"2024-04-23T18:36:02.765732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:45.741648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'13","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765471+0000","last_change":"2024-04-23T18:34:55.298010+0000","last_active":"2024-04-23T18:36:02.765471+0000","last_peered":"2024-04-23T18:36:02.765471+0000","last_clean":"2024-04-23T18:36:02.765471+0000","last_became_active":"2024-04-23T18:34:55.288221+0000","last_became_peered":"2024-04-23T18:34:55.288221+0000","last_unstale":"2024-04-23T18:36:02.765471+0000","last_undegraded":"2024-04-23T18:36:02.765471+0000","last_fullsized":"2024-04-23T18:36:02.765471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:21:35.469676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764723+0000","last_change":"2024-04-23T18:34:01.230435+0000","last_active":"2024-04-23T18:36:02.764723+0000","last_peered":"2024-04-23T18:36:02.764723+0000","last_clean":"2024-04-23T18:36:02.764723+0000","last_became_active":"2024-04-23T18:34:01.230095+0000","last_became_peered":"2024-04-23T18:34:01.230095+0000","last_unstale":"2024-04-23T18:36:02.764723+0000","last_undegraded":"2024-04-23T18:36:02.764723+0000","last_fullsized":"2024-04-23T18:36:02.764723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:31.653254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765399+0000","last_change":"2024-04-23T18:34:00.227853+0000","last_active":"2024-04-23T18:36:02.765399+0000","last_peered":"2024-04-23T18:36:02.765399+0000","last_clean":"2024-04-23T18:36:02.765399+0000","last_became_active":"2024-04-23T18:34:00.226596+0000","last_became_peered":"2024-04-23T18:34:00.226596+0000","last_unstale":"2024-04-23T18:36:02.765399+0000","last_undegraded":"2024-04-23T18:36:02.765399+0000","last_fullsized":"2024-04-23T18:36:02.765399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:16:10.077969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765100+0000","last_change":"2024-04-23T18:34:01.235955+0000","last_active":"2024-04-23T18:36:02.765100+0000","last_peered":"2024-04-23T18:36:02.765100+0000","last_clean":"2024-04-23T18:36:02.765100+0000","last_became_active":"2024-04-23T18:34:01.235609+0000","last_became_peered":"2024-04-23T18:34:01.235609+0000","last_unstale":"2024-04-23T18:36:02.765100+0000","last_undegraded":"2024-04-23T18:36:02.765100+0000","last_fullsized":"2024-04-23T18:36:02.765100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:07:39.480213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764707+0000","last_change":"2024-04-23T18:34:00.226319+0000","last_active":"2024-04-23T18:36:02.764707+0000","last_peered":"2024-04-23T18:36:02.764707+0000","last_clean":"2024-04-23T18:36:02.764707+0000","last_became_active":"2024-04-23T18:34:00.225796+0000","last_became_peered":"2024-04-23T18:34:00.225796+0000","last_unstale":"2024-04-23T18:36:02.764707+0000","last_undegraded":"2024-04-23T18:36:02.764707+0000","last_fullsized":"2024-04-23T18:36:02.764707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:11:18.309181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765238+0000","last_change":"2024-04-23T18:34:01.227336+0000","last_active":"2024-04-23T18:36:02.765238+0000","last_peered":"2024-04-23T18:36:02.765238+0000","last_clean":"2024-04-23T18:36:02.765238+0000","last_became_active":"2024-04-23T18:34:01.226987+0000","last_became_peered":"2024-04-23T18:34:01.226987+0000","last_unstale":"2024-04-23T18:36:02.765238+0000","last_undegraded":"2024-04-23T18:36:02.765238+0000","last_fullsized":"2024-04-23T18:36:02.765238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:22:30.058673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732492+0000","last_change":"2024-04-23T18:34:55.294322+0000","last_active":"2024-04-23T18:36:02.732492+0000","last_peered":"2024-04-23T18:36:02.732492+0000","last_clean":"2024-04-23T18:36:02.732492+0000","last_became_active":"2024-04-23T18:34:55.288855+0000","last_became_peered":"2024-04-23T18:34:55.288855+0000","last_unstale":"2024-04-23T18:36:02.732492+0000","last_undegraded":"2024-04-23T18:36:02.732492+0000","last_fullsized":"2024-04-23T18:36:02.732492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:07:50.097696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764793+0000","last_change":"2024-04-23T18:34:55.287339+0000","last_active":"2024-04-23T18:36:02.764793+0000","last_peered":"2024-04-23T18:36:02.764793+0000","last_clean":"2024-04-23T18:36:02.764793+0000","last_became_active":"2024-04-23T18:34:55.287062+0000","last_became_peered":"2024-04-23T18:34:55.287062+0000","last_unstale":"2024-04-23T18:36:02.764793+0000","last_undegraded":"2024-04-23T18:36:02.764793+0000","last_fullsized":"2024-04-23T18:36:02.764793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:44.864600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731488+0000","last_change":"2024-04-23T18:34:00.227595+0000","last_active":"2024-04-23T18:36:02.731488+0000","last_peered":"2024-04-23T18:36:02.731488+0000","last_clean":"2024-04-23T18:36:02.731488+0000","last_became_active":"2024-04-23T18:34:00.226830+0000","last_became_peered":"2024-04-23T18:34:00.226830+0000","last_unstale":"2024-04-23T18:36:02.731488+0000","last_undegraded":"2024-04-23T18:36:02.731488+0000","last_fullsized":"2024-04-23T18:36:02.731488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:20.661984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764758+0000","last_change":"2024-04-23T18:34:01.225053+0000","last_active":"2024-04-23T18:36:02.764758+0000","last_peered":"2024-04-23T18:36:02.764758+0000","last_clean":"2024-04-23T18:36:02.764758+0000","last_became_active":"2024-04-23T18:34:01.224723+0000","last_became_peered":"2024-04-23T18:34:01.224723+0000","last_unstale":"2024-04-23T18:36:02.764758+0000","last_undegraded":"2024-04-23T18:36:02.764758+0000","last_fullsized":"2024-04-23T18:36:02.764758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:01:51.486691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764506+0000","last_change":"2024-04-23T18:34:00.224783+0000","last_active":"2024-04-23T18:36:02.764506+0000","last_peered":"2024-04-23T18:36:02.764506+0000","last_clean":"2024-04-23T18:36:02.764506+0000","last_became_active":"2024-04-23T18:34:00.224436+0000","last_became_peered":"2024-04-23T18:34:00.224436+0000","last_unstale":"2024-04-23T18:36:02.764506+0000","last_undegraded":"2024-04-23T18:36:02.764506+0000","last_fullsized":"2024-04-23T18:36:02.764506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:33.829188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.721558+0000","last_change":"2024-04-23T18:34:01.224283+0000","last_active":"2024-04-23T18:36:02.721558+0000","last_peered":"2024-04-23T18:36:02.721558+0000","last_clean":"2024-04-23T18:36:02.721558+0000","last_became_active":"2024-04-23T18:34:01.223949+0000","last_became_peered":"2024-04-23T18:34:01.223949+0000","last_unstale":"2024-04-23T18:36:02.721558+0000","last_undegraded":"2024-04-23T18:36:02.721558+0000","last_fullsized":"2024-04-23T18:36:02.721558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:50.560687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":268,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.690843+0000","last_change":"2024-04-23T18:34:00.218453+0000","last_active":"2024-04-23T18:36:03.690843+0000","last_peered":"2024-04-23T18:36:03.690843+0000","last_clean":"2024-04-23T18:36:03.690843+0000","last_became_active":"2024-04-23T18:34:00.218036+0000","last_became_peered":"2024-04-23T18:34:00.218036+0000","last_unstale":"2024-04-23T18:36:03.690843+0000","last_undegraded":"2024-04-23T18:36:03.690843+0000","last_fullsized":"2024-04-23T18:36:03.690843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:49.485185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764932+0000","last_change":"2024-04-23T18:34:01.231143+0000","last_active":"2024-04-23T18:36:02.764932+0000","last_peered":"2024-04-23T18:36:02.764932+0000","last_clean":"2024-04-23T18:36:02.764932+0000","last_became_active":"2024-04-23T18:34:01.230812+0000","last_became_peered":"2024-04-23T18:34:01.230812+0000","last_unstale":"2024-04-23T18:36:02.764932+0000","last_undegraded":"2024-04-23T18:36:02.764932+0000","last_fullsized":"2024-04-23T18:36:02.764932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:28.704807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765595+0000","last_change":"2024-04-23T18:34:00.235711+0000","last_active":"2024-04-23T18:36:02.765595+0000","last_peered":"2024-04-23T18:36:02.765595+0000","last_clean":"2024-04-23T18:36:02.765595+0000","last_became_active":"2024-04-23T18:34:00.235333+0000","last_became_peered":"2024-04-23T18:34:00.235333+0000","last_unstale":"2024-04-23T18:36:02.765595+0000","last_undegraded":"2024-04-23T18:36:02.765595+0000","last_fullsized":"2024-04-23T18:36:02.765595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:51.320744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765599+0000","last_change":"2024-04-23T18:34:01.238291+0000","last_active":"2024-04-23T18:36:02.765599+0000","last_peered":"2024-04-23T18:36:02.765599+0000","last_clean":"2024-04-23T18:36:02.765599+0000","last_became_active":"2024-04-23T18:34:01.237972+0000","last_became_peered":"2024-04-23T18:34:01.237972+0000","last_unstale":"2024-04-23T18:36:02.765599+0000","last_undegraded":"2024-04-23T18:36:02.765599+0000","last_fullsized":"2024-04-23T18:36:02.765599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:36:17.108170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765280+0000","last_change":"2024-04-23T18:34:00.227249+0000","last_active":"2024-04-23T18:36:02.765280+0000","last_peered":"2024-04-23T18:36:02.765280+0000","last_clean":"2024-04-23T18:36:02.765280+0000","last_became_active":"2024-04-23T18:34:00.226854+0000","last_became_peered":"2024-04-23T18:34:00.226854+0000","last_unstale":"2024-04-23T18:36:02.765280+0000","last_undegraded":"2024-04-23T18:36:02.765280+0000","last_fullsized":"2024-04-23T18:36:02.765280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:49.619906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764813+0000","last_change":"2024-04-23T18:34:01.225518+0000","last_active":"2024-04-23T18:36:02.764813+0000","last_peered":"2024-04-23T18:36:02.764813+0000","last_clean":"2024-04-23T18:36:02.764813+0000","last_became_active":"2024-04-23T18:34:01.224907+0000","last_became_peered":"2024-04-23T18:34:01.224907+0000","last_unstale":"2024-04-23T18:36:02.764813+0000","last_undegraded":"2024-04-23T18:36:02.764813+0000","last_fullsized":"2024-04-23T18:36:02.764813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:19.876629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":307,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.690858+0000","last_change":"2024-04-23T18:34:00.224770+0000","last_active":"2024-04-23T18:36:03.690858+0000","last_peered":"2024-04-23T18:36:03.690858+0000","last_clean":"2024-04-23T18:36:03.690858+0000","last_became_active":"2024-04-23T18:34:00.224395+0000","last_became_peered":"2024-04-23T18:34:00.224395+0000","last_unstale":"2024-04-23T18:36:03.690858+0000","last_undegraded":"2024-04-23T18:36:03.690858+0000","last_fullsized":"2024-04-23T18:36:03.690858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:14:09.172479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60022,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":248,"num_read_kb":183,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765184+0000","last_change":"2024-04-23T18:34:01.227771+0000","last_active":"2024-04-23T18:36:02.765184+0000","last_peered":"2024-04-23T18:36:02.765184+0000","last_clean":"2024-04-23T18:36:02.765184+0000","last_became_active":"2024-04-23T18:34:01.225063+0000","last_became_peered":"2024-04-23T18:34:01.225063+0000","last_unstale":"2024-04-23T18:36:02.765184+0000","last_undegraded":"2024-04-23T18:36:02.765184+0000","last_fullsized":"2024-04-23T18:36:02.765184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:53.921587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722785+0000","last_change":"2024-04-23T18:34:00.230455+0000","last_active":"2024-04-23T18:36:02.722785+0000","last_peered":"2024-04-23T18:36:02.722785+0000","last_clean":"2024-04-23T18:36:02.722785+0000","last_became_active":"2024-04-23T18:34:00.229934+0000","last_became_peered":"2024-04-23T18:34:00.229934+0000","last_unstale":"2024-04-23T18:36:02.722785+0000","last_undegraded":"2024-04-23T18:36:02.722785+0000","last_fullsized":"2024-04-23T18:36:02.722785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:26:34.787606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731340+0000","last_change":"2024-04-23T18:34:01.225757+0000","last_active":"2024-04-23T18:36:02.731340+0000","last_peered":"2024-04-23T18:36:02.731340+0000","last_clean":"2024-04-23T18:36:02.731340+0000","last_became_active":"2024-04-23T18:34:01.225447+0000","last_became_peered":"2024-04-23T18:34:01.225447+0000","last_unstale":"2024-04-23T18:36:02.731340+0000","last_undegraded":"2024-04-23T18:36:02.731340+0000","last_fullsized":"2024-04-23T18:36:02.731340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:17.212135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764698+0000","last_change":"2024-04-23T18:34:00.231749+0000","last_active":"2024-04-23T18:36:02.764698+0000","last_peered":"2024-04-23T18:36:02.764698+0000","last_clean":"2024-04-23T18:36:02.764698+0000","last_became_active":"2024-04-23T18:34:00.231207+0000","last_became_peered":"2024-04-23T18:34:00.231207+0000","last_unstale":"2024-04-23T18:36:02.764698+0000","last_undegraded":"2024-04-23T18:36:02.764698+0000","last_fullsized":"2024-04-23T18:36:02.764698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:23:05.723265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765316+0000","last_change":"2024-04-23T18:35:55.690568+0000","last_active":"2024-04-23T18:36:02.765316+0000","last_peered":"2024-04-23T18:36:02.765316+0000","last_clean":"2024-04-23T18:36:02.765316+0000","last_became_active":"2024-04-23T18:35:55.690174+0000","last_became_peered":"2024-04-23T18:35:55.690174+0000","last_unstale":"2024-04-23T18:36:02.765316+0000","last_undegraded":"2024-04-23T18:36:02.765316+0000","last_fullsized":"2024-04-23T18:36:02.765316+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:29.462962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.734130+0000","last_change":"2024-04-23T18:34:00.228578+0000","last_active":"2024-04-23T18:36:03.734130+0000","last_peered":"2024-04-23T18:36:03.734130+0000","last_clean":"2024-04-23T18:36:03.734130+0000","last_became_active":"2024-04-23T18:34:00.228238+0000","last_became_peered":"2024-04-23T18:34:00.228238+0000","last_unstale":"2024-04-23T18:36:03.734130+0000","last_undegraded":"2024-04-23T18:36:03.734130+0000","last_fullsized":"2024-04-23T18:36:03.734130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:58:21.283593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764889+0000","last_change":"2024-04-23T18:34:01.239481+0000","last_active":"2024-04-23T18:36:02.764889+0000","last_peered":"2024-04-23T18:36:02.764889+0000","last_clean":"2024-04-23T18:36:02.764889+0000","last_became_active":"2024-04-23T18:34:01.239042+0000","last_became_peered":"2024-04-23T18:34:01.239042+0000","last_unstale":"2024-04-23T18:36:02.764889+0000","last_undegraded":"2024-04-23T18:36:02.764889+0000","last_fullsized":"2024-04-23T18:36:02.764889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:45.755462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734712+0000","last_change":"2024-04-23T18:34:55.273529+0000","last_active":"2024-04-23T18:36:02.734712+0000","last_peered":"2024-04-23T18:36:02.734712+0000","last_clean":"2024-04-23T18:36:02.734712+0000","last_became_active":"2024-04-23T18:34:55.259780+0000","last_became_peered":"2024-04-23T18:34:55.259780+0000","last_unstale":"2024-04-23T18:36:02.734712+0000","last_undegraded":"2024-04-23T18:36:02.734712+0000","last_fullsized":"2024-04-23T18:36:02.734712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:48:56.272606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764515+0000","last_change":"2024-04-23T18:34:01.239943+0000","last_active":"2024-04-23T18:36:02.764515+0000","last_peered":"2024-04-23T18:36:02.764515+0000","last_clean":"2024-04-23T18:36:02.764515+0000","last_became_active":"2024-04-23T18:34:01.239588+0000","last_became_peered":"2024-04-23T18:34:01.239588+0000","last_unstale":"2024-04-23T18:36:02.764515+0000","last_undegraded":"2024-04-23T18:36:02.764515+0000","last_fullsized":"2024-04-23T18:36:02.764515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:51:39.344341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765578+0000","last_change":"2024-04-23T18:34:00.227372+0000","last_active":"2024-04-23T18:36:02.765578+0000","last_peered":"2024-04-23T18:36:02.765578+0000","last_clean":"2024-04-23T18:36:02.765578+0000","last_became_active":"2024-04-23T18:34:00.226709+0000","last_became_peered":"2024-04-23T18:34:00.226709+0000","last_unstale":"2024-04-23T18:36:02.765578+0000","last_undegraded":"2024-04-23T18:36:02.765578+0000","last_fullsized":"2024-04-23T18:36:02.765578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:20.047108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734890+0000","last_change":"2024-04-23T18:34:01.235389+0000","last_active":"2024-04-23T18:36:02.734890+0000","last_peered":"2024-04-23T18:36:02.734890+0000","last_clean":"2024-04-23T18:36:02.734890+0000","last_became_active":"2024-04-23T18:34:01.235026+0000","last_became_peered":"2024-04-23T18:34:01.235026+0000","last_unstale":"2024-04-23T18:36:02.734890+0000","last_undegraded":"2024-04-23T18:36:02.734890+0000","last_fullsized":"2024-04-23T18:36:02.734890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:48.849525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.721668+0000","last_change":"2024-04-23T18:34:55.260852+0000","last_active":"2024-04-23T18:36:02.721668+0000","last_peered":"2024-04-23T18:36:02.721668+0000","last_clean":"2024-04-23T18:36:02.721668+0000","last_became_active":"2024-04-23T18:34:55.260452+0000","last_became_peered":"2024-04-23T18:34:55.260452+0000","last_unstale":"2024-04-23T18:36:02.721668+0000","last_undegraded":"2024-04-23T18:36:02.721668+0000","last_fullsized":"2024-04-23T18:36:02.721668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:28:13.496447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732819+0000","last_change":"2024-04-23T18:34:01.224992+0000","last_active":"2024-04-23T18:36:02.732819+0000","last_peered":"2024-04-23T18:36:02.732819+0000","last_clean":"2024-04-23T18:36:02.732819+0000","last_became_active":"2024-04-23T18:34:01.224508+0000","last_became_peered":"2024-04-23T18:34:01.224508+0000","last_unstale":"2024-04-23T18:36:02.732819+0000","last_undegraded":"2024-04-23T18:36:02.732819+0000","last_fullsized":"2024-04-23T18:36:02.732819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:24.183451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":152,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.700066+0000","last_change":"2024-04-23T18:34:00.234818+0000","last_active":"2024-04-23T18:36:03.700066+0000","last_peered":"2024-04-23T18:36:03.700066+0000","last_clean":"2024-04-23T18:36:03.700066+0000","last_became_active":"2024-04-23T18:34:00.234329+0000","last_became_peered":"2024-04-23T18:34:00.234329+0000","last_unstale":"2024-04-23T18:36:03.700066+0000","last_undegraded":"2024-04-23T18:36:03.700066+0000","last_fullsized":"2024-04-23T18:36:03.700066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:43:02.405694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732868+0000","last_change":"2024-04-23T18:34:55.261073+0000","last_active":"2024-04-23T18:36:02.732868+0000","last_peered":"2024-04-23T18:36:02.732868+0000","last_clean":"2024-04-23T18:36:02.732868+0000","last_became_active":"2024-04-23T18:34:55.260655+0000","last_became_peered":"2024-04-23T18:34:55.260655+0000","last_unstale":"2024-04-23T18:36:02.732868+0000","last_undegraded":"2024-04-23T18:36:02.732868+0000","last_fullsized":"2024-04-23T18:36:02.732868+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:37:36.797785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764638+0000","last_change":"2024-04-23T18:34:01.234184+0000","last_active":"2024-04-23T18:36:02.764638+0000","last_peered":"2024-04-23T18:36:02.764638+0000","last_clean":"2024-04-23T18:36:02.764638+0000","last_became_active":"2024-04-23T18:34:01.233846+0000","last_became_peered":"2024-04-23T18:34:01.233846+0000","last_unstale":"2024-04-23T18:36:02.764638+0000","last_undegraded":"2024-04-23T18:36:02.764638+0000","last_fullsized":"2024-04-23T18:36:02.764638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:41.771495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.960951+0000","last_change":"2024-04-23T18:34:00.227862+0000","last_active":"2024-04-23T18:36:02.960951+0000","last_peered":"2024-04-23T18:36:02.960951+0000","last_clean":"2024-04-23T18:36:02.960951+0000","last_became_active":"2024-04-23T18:34:00.227515+0000","last_became_peered":"2024-04-23T18:34:00.227515+0000","last_unstale":"2024-04-23T18:36:02.960951+0000","last_undegraded":"2024-04-23T18:36:02.960951+0000","last_fullsized":"2024-04-23T18:36:02.960951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:18.595690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765280+0000","last_change":"2024-04-23T18:34:01.236338+0000","last_active":"2024-04-23T18:36:02.765280+0000","last_peered":"2024-04-23T18:36:02.765280+0000","last_clean":"2024-04-23T18:36:02.765280+0000","last_became_active":"2024-04-23T18:34:01.235795+0000","last_became_peered":"2024-04-23T18:34:01.235795+0000","last_unstale":"2024-04-23T18:36:02.765280+0000","last_undegraded":"2024-04-23T18:36:02.765280+0000","last_fullsized":"2024-04-23T18:36:02.765280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:45:01.928793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.725362+0000","last_change":"2024-04-23T18:34:00.221851+0000","last_active":"2024-04-23T18:36:02.725362+0000","last_peered":"2024-04-23T18:36:02.725362+0000","last_clean":"2024-04-23T18:36:02.725362+0000","last_became_active":"2024-04-23T18:34:00.221513+0000","last_became_peered":"2024-04-23T18:34:00.221513+0000","last_unstale":"2024-04-23T18:36:02.725362+0000","last_undegraded":"2024-04-23T18:36:02.725362+0000","last_fullsized":"2024-04-23T18:36:02.725362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:15:04.608759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764979+0000","last_change":"2024-04-23T18:32:48.218931+0000","last_active":"2024-04-23T18:36:02.764979+0000","last_peered":"2024-04-23T18:36:02.764979+0000","last_clean":"2024-04-23T18:36:02.764979+0000","last_became_active":"2024-04-23T18:32:48.218484+0000","last_became_peered":"2024-04-23T18:32:48.218484+0000","last_unstale":"2024-04-23T18:36:02.764979+0000","last_undegraded":"2024-04-23T18:36:02.764979+0000","last_fullsized":"2024-04-23T18:36:02.764979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_clean_scrub_stamp":"2024-04-23T18:29:21.029319+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:03:42.234165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731244+0000","last_change":"2024-04-23T18:34:01.220346+0000","last_active":"2024-04-23T18:36:02.731244+0000","last_peered":"2024-04-23T18:36:02.731244+0000","last_clean":"2024-04-23T18:36:02.731244+0000","last_became_active":"2024-04-23T18:34:01.219991+0000","last_became_peered":"2024-04-23T18:34:01.219991+0000","last_unstale":"2024-04-23T18:36:02.731244+0000","last_undegraded":"2024-04-23T18:36:02.731244+0000","last_fullsized":"2024-04-23T18:36:02.731244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:29:00.662700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765123+0000","last_change":"2024-04-23T18:34:00.229625+0000","last_active":"2024-04-23T18:36:02.765123+0000","last_peered":"2024-04-23T18:36:02.765123+0000","last_clean":"2024-04-23T18:36:02.765123+0000","last_became_active":"2024-04-23T18:34:00.229204+0000","last_became_peered":"2024-04-23T18:34:00.229204+0000","last_unstale":"2024-04-23T18:36:02.765123+0000","last_undegraded":"2024-04-23T18:36:02.765123+0000","last_fullsized":"2024-04-23T18:36:02.765123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:42.370870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765328+0000","last_change":"2024-04-23T18:34:00.236000+0000","last_active":"2024-04-23T18:36:02.765328+0000","last_peered":"2024-04-23T18:36:02.765328+0000","last_clean":"2024-04-23T18:36:02.765328+0000","last_became_active":"2024-04-23T18:34:00.235561+0000","last_became_peered":"2024-04-23T18:34:00.235561+0000","last_unstale":"2024-04-23T18:36:02.765328+0000","last_undegraded":"2024-04-23T18:36:02.765328+0000","last_fullsized":"2024-04-23T18:36:02.765328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:07.565489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731441+0000","last_change":"2024-04-23T18:34:01.227955+0000","last_active":"2024-04-23T18:36:02.731441+0000","last_peered":"2024-04-23T18:36:02.731441+0000","last_clean":"2024-04-23T18:36:02.731441+0000","last_became_active":"2024-04-23T18:34:01.225992+0000","last_became_peered":"2024-04-23T18:34:01.225992+0000","last_unstale":"2024-04-23T18:36:02.731441+0000","last_undegraded":"2024-04-23T18:36:02.731441+0000","last_fullsized":"2024-04-23T18:36:02.731441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:05:55.147008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'48","reported_seq":96,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.925135+0000","last_change":"2024-04-23T18:34:00.230133+0000","last_active":"2024-04-23T18:36:02.925135+0000","last_peered":"2024-04-23T18:36:02.925135+0000","last_clean":"2024-04-23T18:36:02.925135+0000","last_became_active":"2024-04-23T18:34:00.229596+0000","last_became_peered":"2024-04-23T18:34:00.229596+0000","last_unstale":"2024-04-23T18:36:02.925135+0000","last_undegraded":"2024-04-23T18:36:02.925135+0000","last_fullsized":"2024-04-23T18:36:02.925135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:23:06.064313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765529+0000","last_change":"2024-04-23T18:34:01.229066+0000","last_active":"2024-04-23T18:36:02.765529+0000","last_peered":"2024-04-23T18:36:02.765529+0000","last_clean":"2024-04-23T18:36:02.765529+0000","last_became_active":"2024-04-23T18:34:01.228711+0000","last_became_peered":"2024-04-23T18:34:01.228711+0000","last_unstale":"2024-04-23T18:36:02.765529+0000","last_undegraded":"2024-04-23T18:36:02.765529+0000","last_fullsized":"2024-04-23T18:36:02.765529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:58:30.601270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.725508+0000","last_change":"2024-04-23T18:34:00.231227+0000","last_active":"2024-04-23T18:36:02.725508+0000","last_peered":"2024-04-23T18:36:02.725508+0000","last_clean":"2024-04-23T18:36:02.725508+0000","last_became_active":"2024-04-23T18:34:00.230843+0000","last_became_peered":"2024-04-23T18:34:00.230843+0000","last_unstale":"2024-04-23T18:36:02.725508+0000","last_undegraded":"2024-04-23T18:36:02.725508+0000","last_fullsized":"2024-04-23T18:36:02.725508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:55.416591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765173+0000","last_change":"2024-04-23T18:34:01.233164+0000","last_active":"2024-04-23T18:36:02.765173+0000","last_peered":"2024-04-23T18:36:02.765173+0000","last_clean":"2024-04-23T18:36:02.765173+0000","last_became_active":"2024-04-23T18:34:01.232827+0000","last_became_peered":"2024-04-23T18:34:01.232827+0000","last_unstale":"2024-04-23T18:36:02.765173+0000","last_undegraded":"2024-04-23T18:36:02.765173+0000","last_fullsized":"2024-04-23T18:36:02.765173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:09.635515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764498+0000","last_change":"2024-04-23T18:34:00.225651+0000","last_active":"2024-04-23T18:36:02.764498+0000","last_peered":"2024-04-23T18:36:02.764498+0000","last_clean":"2024-04-23T18:36:02.764498+0000","last_became_active":"2024-04-23T18:34:00.225001+0000","last_became_peered":"2024-04-23T18:34:00.225001+0000","last_unstale":"2024-04-23T18:36:02.764498+0000","last_undegraded":"2024-04-23T18:36:02.764498+0000","last_fullsized":"2024-04-23T18:36:02.764498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:27:44.265388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765100+0000","last_change":"2024-04-23T18:34:01.224131+0000","last_active":"2024-04-23T18:36:02.765100+0000","last_peered":"2024-04-23T18:36:02.765100+0000","last_clean":"2024-04-23T18:36:02.765100+0000","last_became_active":"2024-04-23T18:34:01.223742+0000","last_became_peered":"2024-04-23T18:34:01.223742+0000","last_unstale":"2024-04-23T18:36:02.765100+0000","last_undegraded":"2024-04-23T18:36:02.765100+0000","last_fullsized":"2024-04-23T18:36:02.765100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:29:26.430964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764840+0000","last_change":"2024-04-23T18:34:00.235789+0000","last_active":"2024-04-23T18:36:02.764840+0000","last_peered":"2024-04-23T18:36:02.764840+0000","last_clean":"2024-04-23T18:36:02.764840+0000","last_became_active":"2024-04-23T18:34:00.235452+0000","last_became_peered":"2024-04-23T18:34:00.235452+0000","last_unstale":"2024-04-23T18:36:02.764840+0000","last_undegraded":"2024-04-23T18:36:02.764840+0000","last_fullsized":"2024-04-23T18:36:02.764840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:21.090121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734787+0000","last_change":"2024-04-23T18:34:01.226156+0000","last_active":"2024-04-23T18:36:02.734787+0000","last_peered":"2024-04-23T18:36:02.734787+0000","last_clean":"2024-04-23T18:36:02.734787+0000","last_became_active":"2024-04-23T18:34:01.225817+0000","last_became_peered":"2024-04-23T18:34:01.225817+0000","last_unstale":"2024-04-23T18:36:02.734787+0000","last_undegraded":"2024-04-23T18:36:02.734787+0000","last_fullsized":"2024-04-23T18:36:02.734787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:36.337139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.723101+0000","last_change":"2024-04-23T18:34:00.235116+0000","last_active":"2024-04-23T18:36:02.723101+0000","last_peered":"2024-04-23T18:36:02.723101+0000","last_clean":"2024-04-23T18:36:02.723101+0000","last_became_active":"2024-04-23T18:34:00.234521+0000","last_became_peered":"2024-04-23T18:34:00.234521+0000","last_unstale":"2024-04-23T18:36:02.723101+0000","last_undegraded":"2024-04-23T18:36:02.723101+0000","last_fullsized":"2024-04-23T18:36:02.723101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:51.061164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765111+0000","last_change":"2024-04-23T18:34:01.226445+0000","last_active":"2024-04-23T18:36:02.765111+0000","last_peered":"2024-04-23T18:36:02.765111+0000","last_clean":"2024-04-23T18:36:02.765111+0000","last_became_active":"2024-04-23T18:34:01.225875+0000","last_became_peered":"2024-04-23T18:34:01.225875+0000","last_unstale":"2024-04-23T18:36:02.765111+0000","last_undegraded":"2024-04-23T18:36:02.765111+0000","last_fullsized":"2024-04-23T18:36:02.765111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:48:38.712502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765246+0000","last_change":"2024-04-23T18:34:00.231918+0000","last_active":"2024-04-23T18:36:02.765246+0000","last_peered":"2024-04-23T18:36:02.765246+0000","last_clean":"2024-04-23T18:36:02.765246+0000","last_became_active":"2024-04-23T18:34:00.231550+0000","last_became_peered":"2024-04-23T18:34:00.231550+0000","last_unstale":"2024-04-23T18:36:02.765246+0000","last_undegraded":"2024-04-23T18:36:02.765246+0000","last_fullsized":"2024-04-23T18:36:02.765246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:52.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765279+0000","last_change":"2024-04-23T18:34:01.236360+0000","last_active":"2024-04-23T18:36:02.765279+0000","last_peered":"2024-04-23T18:36:02.765279+0000","last_clean":"2024-04-23T18:36:02.765279+0000","last_became_active":"2024-04-23T18:34:01.236056+0000","last_became_peered":"2024-04-23T18:34:01.236056+0000","last_unstale":"2024-04-23T18:36:02.765279+0000","last_undegraded":"2024-04-23T18:36:02.765279+0000","last_fullsized":"2024-04-23T18:36:02.765279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:54.535158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'9","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.736528+0000","last_change":"2024-04-23T18:34:00.233159+0000","last_active":"2024-04-23T18:36:03.736528+0000","last_peered":"2024-04-23T18:36:03.736528+0000","last_clean":"2024-04-23T18:36:03.736528+0000","last_became_active":"2024-04-23T18:34:00.232728+0000","last_became_peered":"2024-04-23T18:34:00.232728+0000","last_unstale":"2024-04-23T18:36:03.736528+0000","last_undegraded":"2024-04-23T18:36:03.736528+0000","last_fullsized":"2024-04-23T18:36:03.736528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:23.498057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2484,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765647+0000","last_change":"2024-04-23T18:34:01.226134+0000","last_active":"2024-04-23T18:36:02.765647+0000","last_peered":"2024-04-23T18:36:02.765647+0000","last_clean":"2024-04-23T18:36:02.765647+0000","last_became_active":"2024-04-23T18:34:01.225759+0000","last_became_peered":"2024-04-23T18:34:01.225759+0000","last_unstale":"2024-04-23T18:36:02.765647+0000","last_undegraded":"2024-04-23T18:36:02.765647+0000","last_fullsized":"2024-04-23T18:36:02.765647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:24.770595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.923105+0000","last_change":"2024-04-23T18:34:00.230054+0000","last_active":"2024-04-23T18:36:02.923105+0000","last_peered":"2024-04-23T18:36:02.923105+0000","last_clean":"2024-04-23T18:36:02.923105+0000","last_became_active":"2024-04-23T18:34:00.229621+0000","last_became_peered":"2024-04-23T18:34:00.229621+0000","last_unstale":"2024-04-23T18:36:02.923105+0000","last_undegraded":"2024-04-23T18:36:02.923105+0000","last_fullsized":"2024-04-23T18:36:02.923105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:13:59.317282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765433+0000","last_change":"2024-04-23T18:34:01.240053+0000","last_active":"2024-04-23T18:36:02.765433+0000","last_peered":"2024-04-23T18:36:02.765433+0000","last_clean":"2024-04-23T18:36:02.765433+0000","last_became_active":"2024-04-23T18:34:01.239716+0000","last_became_peered":"2024-04-23T18:34:01.239716+0000","last_unstale":"2024-04-23T18:36:02.765433+0000","last_undegraded":"2024-04-23T18:36:02.765433+0000","last_fullsized":"2024-04-23T18:36:02.765433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:54.874374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731781+0000","last_change":"2024-04-23T18:34:00.227914+0000","last_active":"2024-04-23T18:36:02.731781+0000","last_peered":"2024-04-23T18:36:02.731781+0000","last_clean":"2024-04-23T18:36:02.731781+0000","last_became_active":"2024-04-23T18:34:00.227540+0000","last_became_peered":"2024-04-23T18:34:00.227540+0000","last_unstale":"2024-04-23T18:36:02.731781+0000","last_undegraded":"2024-04-23T18:36:02.731781+0000","last_fullsized":"2024-04-23T18:36:02.731781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:18.442397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765656+0000","last_change":"2024-04-23T18:34:01.227319+0000","last_active":"2024-04-23T18:36:02.765656+0000","last_peered":"2024-04-23T18:36:02.765656+0000","last_clean":"2024-04-23T18:36:02.765656+0000","last_became_active":"2024-04-23T18:34:01.226969+0000","last_became_peered":"2024-04-23T18:34:01.226969+0000","last_unstale":"2024-04-23T18:36:02.765656+0000","last_undegraded":"2024-04-23T18:36:02.765656+0000","last_fullsized":"2024-04-23T18:36:02.765656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:19.302388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734899+0000","last_change":"2024-04-23T18:34:01.234466+0000","last_active":"2024-04-23T18:36:02.734899+0000","last_peered":"2024-04-23T18:36:02.734899+0000","last_clean":"2024-04-23T18:36:02.734899+0000","last_became_active":"2024-04-23T18:34:01.234127+0000","last_became_peered":"2024-04-23T18:34:01.234127+0000","last_unstale":"2024-04-23T18:36:02.734899+0000","last_undegraded":"2024-04-23T18:36:02.734899+0000","last_fullsized":"2024-04-23T18:36:02.734899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:51:51.912320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722721+0000","last_change":"2024-04-23T18:34:01.229707+0000","last_active":"2024-04-23T18:36:02.722721+0000","last_peered":"2024-04-23T18:36:02.722721+0000","last_clean":"2024-04-23T18:36:02.722721+0000","last_became_active":"2024-04-23T18:34:01.229311+0000","last_became_peered":"2024-04-23T18:34:01.229311+0000","last_unstale":"2024-04-23T18:36:02.722721+0000","last_undegraded":"2024-04-23T18:36:02.722721+0000","last_fullsized":"2024-04-23T18:36:02.722721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:15:51.972436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722618+0000","last_change":"2024-04-23T18:34:01.227285+0000","last_active":"2024-04-23T18:36:02.722618+0000","last_peered":"2024-04-23T18:36:02.722618+0000","last_clean":"2024-04-23T18:36:02.722618+0000","last_became_active":"2024-04-23T18:34:01.226960+0000","last_became_peered":"2024-04-23T18:34:01.226960+0000","last_unstale":"2024-04-23T18:36:02.722618+0000","last_undegraded":"2024-04-23T18:36:02.722618+0000","last_fullsized":"2024-04-23T18:36:02.722618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:50.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764963+0000","last_change":"2024-04-23T18:34:01.230697+0000","last_active":"2024-04-23T18:36:02.764963+0000","last_peered":"2024-04-23T18:36:02.764963+0000","last_clean":"2024-04-23T18:36:02.764963+0000","last_became_active":"2024-04-23T18:34:01.230352+0000","last_became_peered":"2024-04-23T18:34:01.230352+0000","last_unstale":"2024-04-23T18:36:02.764963+0000","last_undegraded":"2024-04-23T18:36:02.764963+0000","last_fullsized":"2024-04-23T18:36:02.764963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:35:10.609773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731598+0000","last_change":"2024-04-23T18:34:01.226970+0000","last_active":"2024-04-23T18:36:02.731598+0000","last_peered":"2024-04-23T18:36:02.731598+0000","last_clean":"2024-04-23T18:36:02.731598+0000","last_became_active":"2024-04-23T18:34:01.224151+0000","last_became_peered":"2024-04-23T18:34:01.224151+0000","last_unstale":"2024-04-23T18:36:02.731598+0000","last_undegraded":"2024-04-23T18:36:02.731598+0000","last_fullsized":"2024-04-23T18:36:02.731598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:36:15.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734930+0000","last_change":"2024-04-23T18:34:01.234194+0000","last_active":"2024-04-23T18:36:02.734930+0000","last_peered":"2024-04-23T18:36:02.734930+0000","last_clean":"2024-04-23T18:36:02.734930+0000","last_became_active":"2024-04-23T18:34:01.232404+0000","last_became_peered":"2024-04-23T18:34:01.232404+0000","last_unstale":"2024-04-23T18:36:02.734930+0000","last_undegraded":"2024-04-23T18:36:02.734930+0000","last_fullsized":"2024-04-23T18:36:02.734930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:30.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722974+0000","last_change":"2024-04-23T18:34:01.219507+0000","last_active":"2024-04-23T18:36:02.722974+0000","last_peered":"2024-04-23T18:36:02.722974+0000","last_clean":"2024-04-23T18:36:02.722974+0000","last_became_active":"2024-04-23T18:34:01.218997+0000","last_became_peered":"2024-04-23T18:34:01.218997+0000","last_unstale":"2024-04-23T18:36:02.722974+0000","last_undegraded":"2024-04-23T18:36:02.722974+0000","last_fullsized":"2024-04-23T18:36:02.722974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:38.233652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765439+0000","last_change":"2024-04-23T18:34:01.231846+0000","last_active":"2024-04-23T18:36:02.765439+0000","last_peered":"2024-04-23T18:36:02.765439+0000","last_clean":"2024-04-23T18:36:02.765439+0000","last_became_active":"2024-04-23T18:34:01.231467+0000","last_became_peered":"2024-04-23T18:34:01.231467+0000","last_unstale":"2024-04-23T18:36:02.765439+0000","last_undegraded":"2024-04-23T18:36:02.765439+0000","last_fullsized":"2024-04-23T18:36:02.765439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:37:27.891180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764528+0000","last_change":"2024-04-23T18:34:01.237086+0000","last_active":"2024-04-23T18:36:02.764528+0000","last_peered":"2024-04-23T18:36:02.764528+0000","last_clean":"2024-04-23T18:36:02.764528+0000","last_became_active":"2024-04-23T18:34:01.236610+0000","last_became_peered":"2024-04-23T18:34:01.236610+0000","last_unstale":"2024-04-23T18:36:02.764528+0000","last_undegraded":"2024-04-23T18:36:02.764528+0000","last_fullsized":"2024-04-23T18:36:02.764528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:24.916553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764832+0000","last_change":"2024-04-23T18:34:01.231091+0000","last_active":"2024-04-23T18:36:02.764832+0000","last_peered":"2024-04-23T18:36:02.764832+0000","last_clean":"2024-04-23T18:36:02.764832+0000","last_became_active":"2024-04-23T18:34:01.230731+0000","last_became_peered":"2024-04-23T18:34:01.230731+0000","last_unstale":"2024-04-23T18:36:02.764832+0000","last_undegraded":"2024-04-23T18:36:02.764832+0000","last_fullsized":"2024-04-23T18:36:02.764832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:22.771515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764631+0000","last_change":"2024-04-23T18:34:01.231820+0000","last_active":"2024-04-23T18:36:02.764631+0000","last_peered":"2024-04-23T18:36:02.764631+0000","last_clean":"2024-04-23T18:36:02.764631+0000","last_became_active":"2024-04-23T18:34:01.230320+0000","last_became_peered":"2024-04-23T18:34:01.230320+0000","last_unstale":"2024-04-23T18:36:02.764631+0000","last_undegraded":"2024-04-23T18:36:02.764631+0000","last_fullsized":"2024-04-23T18:36:02.764631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:27:35.324323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731327+0000","last_change":"2024-04-23T18:34:01.228723+0000","last_active":"2024-04-23T18:36:02.731327+0000","last_peered":"2024-04-23T18:36:02.731327+0000","last_clean":"2024-04-23T18:36:02.731327+0000","last_became_active":"2024-04-23T18:34:01.227500+0000","last_became_peered":"2024-04-23T18:34:01.227500+0000","last_unstale":"2024-04-23T18:36:02.731327+0000","last_undegraded":"2024-04-23T18:36:02.731327+0000","last_fullsized":"2024-04-23T18:36:02.731327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:17.908021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734867+0000","last_change":"2024-04-23T18:34:01.227541+0000","last_active":"2024-04-23T18:36:02.734867+0000","last_peered":"2024-04-23T18:36:02.734867+0000","last_clean":"2024-04-23T18:36:02.734867+0000","last_became_active":"2024-04-23T18:34:01.227201+0000","last_became_peered":"2024-04-23T18:34:01.227201+0000","last_unstale":"2024-04-23T18:36:02.734867+0000","last_undegraded":"2024-04-23T18:36:02.734867+0000","last_fullsized":"2024-04-23T18:36:02.734867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:25:24.886850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107658,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":594196,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1319,"num_read_kb":769,"num_write":371,"num_write_kb":1459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":688128,"data_stored":1782588,"data_compressed":81528,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":488912,"data_compressed":14490,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1171456,"data_stored":1271310,"data_compressed":58406,"data_compressed_allocated":77824,"data_compressed_original":761856,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":881414,"data_compressed":33636,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037813,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":881680,"data_compressed":33636,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":1081543,"data_compressed":50376,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495680,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":488614,"data_compressed":14490,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":549158,"data_compressed":16364,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":489786,"data_compressed":14490,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":550134,"data_compressed":16364,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1012556,"data_compressed":39792,"data_compressed_allocated":53248,"data_compressed_original":557056,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":488167,"data_compressed":14490,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1080913,"data_compressed":50376,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62473,"data_compressed":1874,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63840,"data_compressed":1874,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134952,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":193749,"data_compressed":8030,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:36:08.635 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-23T18:36:08.637 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:36:08.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.779+0000 7f90c7e89640 1 -- 172.21.15.132:0/3502010278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c0107250 msgr2=0x7f90c010ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.779+0000 7f90c7e89640 1 --2- 172.21.15.132:0/3502010278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c0107250 0x7f90c010ee20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009ca0 tx=0x7f90b802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.779+0000 7f90c7e89640 1 -- 172.21.15.132:0/3502010278 shutdown_connections 2024-04-23T18:36:08.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.779+0000 7f90c7e89640 1 --2- 172.21.15.132:0/3502010278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c0102b30 0x7f90c0106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.779+0000 7f90c7e89640 1 --2- 172.21.15.132:0/3502010278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c0107250 0x7f90c010ee20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.779+0000 7f90c7e89640 1 --2- 172.21.15.132:0/3502010278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c0102180 0x7f90c0102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.779+0000 7f90c7e89640 1 -- 172.21.15.132:0/3502010278 >> 172.21.15.132:0/3502010278 conn(0x7f90c0079350 msgr2=0x7f90c0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.779+0000 7f90c7e89640 1 -- 172.21.15.132:0/3502010278 shutdown_connections 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.779+0000 7f90c7e89640 1 -- 172.21.15.132:0/3502010278 wait complete. 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c7e89640 1 Processor -- start 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c7e89640 1 -- start start 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c7e89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c0102180 0x7f90c01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c7e89640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c0107250 0x7f90c01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c7e89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c019a760 0x7f90c019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c7e89640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01121c0 con 0x7f90c0107250 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c7e89640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0112040 con 0x7f90c019a760 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c7e89640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0112340 con 0x7f90c0102180 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c5424640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c0107250 0x7f90c01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:08.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c5424640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c0107250 0x7f90c01a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57834/0 (socket says 172.21.15.132:57834) 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c5424640 1 -- 172.21.15.132:0/1943206313 learned_addr learned my addr 172.21.15.132:0/1943206313 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c6426640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c019a760 0x7f90c019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c5c25640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c0102180 0x7f90c01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c5424640 1 -- 172.21.15.132:0/1943206313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c0102180 msgr2=0x7f90c01a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c5424640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c0102180 0x7f90c01a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c5424640 1 -- 172.21.15.132:0/1943206313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c019a760 msgr2=0x7f90c019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c5424640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c019a760 0x7f90c019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c5424640 1 -- 172.21.15.132:0/1943206313 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c019b360 con 0x7f90c0107250 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.780+0000 7f90c5c25640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c0102180 0x7f90c01a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.781+0000 7f90c5424640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c0107250 0x7f90c01a0be0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f90b000c970 tx=0x7f90b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.781+0000 7f90b6ffd640 1 -- 172.21.15.132:0/1943206313 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0007bb0 con 0x7f90c0107250 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.781+0000 7f90b6ffd640 1 -- 172.21.15.132:0/1943206313 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b0007d50 con 0x7f90c0107250 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.781+0000 7f90c7e89640 1 -- 172.21.15.132:0/1943206313 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c01a7440 con 0x7f90c0107250 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.781+0000 7f90b6ffd640 1 -- 172.21.15.132:0/1943206313 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0018730 con 0x7f90c0107250 2024-04-23T18:36:08.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.781+0000 7f90c7e89640 1 -- 172.21.15.132:0/1943206313 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c01a77f0 con 0x7f90c0107250 2024-04-23T18:36:08.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.782+0000 7f90c7e89640 1 -- 172.21.15.132:0/1943206313 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c0103650 con 0x7f90c0107250 2024-04-23T18:36:08.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.783+0000 7f90b6ffd640 1 -- 172.21.15.132:0/1943206313 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f90b000f990 con 0x7f90c0107250 2024-04-23T18:36:08.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.783+0000 7f90b6ffd640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f90ac0a3e40 0x7f90ac0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:08.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.783+0000 7f90c5c25640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f90ac0a3e40 0x7f90ac0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:08.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.783+0000 7f90b6ffd640 1 -- 172.21.15.132:0/1943206313 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f90b00d0a80 con 0x7f90c0107250 2024-04-23T18:36:08.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.784+0000 7f90c5c25640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f90ac0a3e40 0x7f90ac0a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0058c0 tx=0x7f90bc005850 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:08.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.788+0000 7f90b6ffd640 1 -- 172.21.15.132:0/1943206313 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90b0014030 con 0x7f90c0107250 2024-04-23T18:36:08.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:08 smithi132 ceph-mon[29178]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:36:08.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:08 smithi132 ceph-mon[29178]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:36:08.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:08 smithi132 ceph-mon[29178]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:36:08.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:08 smithi132 ceph-mon[29178]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:36:08.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:08 smithi132 ceph-mon[29178]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:36:08.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:08 smithi132 ceph-mon[29178]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:36:08.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:08 smithi132 ceph-mon[29178]: Updating smithi132:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:36:08.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:08 smithi132 ceph-mon[29178]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-23T18:36:08.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:08 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/28674201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:36:08.862 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:08 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/777435506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:36:08.933 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:08 smithi148 ceph-mon[32311]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:36:08.933 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:08 smithi148 ceph-mon[32311]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:36:08.933 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:08 smithi148 ceph-mon[32311]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:36:08.934 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:08 smithi148 ceph-mon[32311]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:36:08.934 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:08 smithi148 ceph-mon[32311]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:36:08.934 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:08 smithi148 ceph-mon[32311]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:36:08.934 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:08 smithi148 ceph-mon[32311]: Updating smithi132:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:36:08.934 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:08 smithi148 ceph-mon[32311]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-23T18:36:08.934 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:08 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/28674201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:36:08.934 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:08 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/777435506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:36:08.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.963+0000 7f90c7e89640 1 -- 172.21.15.132:0/1943206313 --> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90c0102ba0 con 0x7f90ac0a3e40 2024-04-23T18:36:08.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.970+0000 7f90b6ffd640 1 -- 172.21.15.132:0/1943206313 <== mgr.15366 v2:172.21.15.132:6800/3428544997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258129 (secure 0 0 0) 0x7f90c0102ba0 con 0x7f90ac0a3e40 2024-04-23T18:36:08.964 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:08.965 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-04-23T18:36:08.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.973+0000 7f90c7e89640 1 -- 172.21.15.132:0/1943206313 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f90ac0a3e40 msgr2=0x7f90ac0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.973+0000 7f90c7e89640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f90ac0a3e40 0x7f90ac0a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0058c0 tx=0x7f90bc005850 comp rx=0 tx=0).stop 2024-04-23T18:36:08.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.973+0000 7f90c7e89640 1 -- 172.21.15.132:0/1943206313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c0107250 msgr2=0x7f90c01a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:08.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.973+0000 7f90c7e89640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c0107250 0x7f90c01a0be0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f90b000c970 tx=0x7f90b000ce30 comp rx=0 tx=0).stop 2024-04-23T18:36:08.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.973+0000 7f90c7e89640 1 -- 172.21.15.132:0/1943206313 shutdown_connections 2024-04-23T18:36:08.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.973+0000 7f90c7e89640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f90ac0a3e40 0x7f90ac0a6300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.973+0000 7f90c7e89640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c019a760 0x7f90c019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.973+0000 7f90c7e89640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c0107250 0x7f90c01a0be0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.973+0000 7f90c7e89640 1 --2- 172.21.15.132:0/1943206313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c0102180 0x7f90c01a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:08.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.973+0000 7f90c7e89640 1 -- 172.21.15.132:0/1943206313 >> 172.21.15.132:0/1943206313 conn(0x7f90c0079350 msgr2=0x7f90c0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:08.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.973+0000 7f90c7e89640 1 -- 172.21.15.132:0/1943206313 shutdown_connections 2024-04-23T18:36:08.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:08.973+0000 7f90c7e89640 1 -- 172.21.15.132:0/1943206313 wait complete. 2024-04-23T18:36:08.976 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-23T18:36:08.735856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1381821,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1724,"num_read_kb":1129,"num_write":683,"num_write_kb":5315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1019,"ondisk_log_size":1019,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353336,"kb_used_data":12456,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588488,"statfs":{"total":1151940427776,"available":1151578611712,"internally_reserved":0,"allocated":12754944,"data_stored":9790615,"data_compressed":381444,"data_compressed_allocated":516096,"data_compressed_original":4202496,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":197023,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":269,"num_read_kb":241,"num_write":129,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.991177"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734914+0000","last_change":"2024-04-23T18:34:01.231204+0000","last_active":"2024-04-23T18:36:02.734914+0000","last_peered":"2024-04-23T18:36:02.734914+0000","last_clean":"2024-04-23T18:36:02.734914+0000","last_became_active":"2024-04-23T18:34:01.230836+0000","last_became_peered":"2024-04-23T18:34:01.230836+0000","last_unstale":"2024-04-23T18:36:02.734914+0000","last_undegraded":"2024-04-23T18:36:02.734914+0000","last_fullsized":"2024-04-23T18:36:02.734914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:57.744807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765343+0000","last_change":"2024-04-23T18:34:01.225833+0000","last_active":"2024-04-23T18:36:02.765343+0000","last_peered":"2024-04-23T18:36:02.765343+0000","last_clean":"2024-04-23T18:36:02.765343+0000","last_became_active":"2024-04-23T18:34:01.225098+0000","last_became_peered":"2024-04-23T18:34:01.225098+0000","last_unstale":"2024-04-23T18:36:02.765343+0000","last_undegraded":"2024-04-23T18:36:02.765343+0000","last_fullsized":"2024-04-23T18:36:02.765343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:20.720137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765240+0000","last_change":"2024-04-23T18:34:01.235269+0000","last_active":"2024-04-23T18:36:02.765240+0000","last_peered":"2024-04-23T18:36:02.765240+0000","last_clean":"2024-04-23T18:36:02.765240+0000","last_became_active":"2024-04-23T18:34:01.234921+0000","last_became_peered":"2024-04-23T18:34:01.234921+0000","last_unstale":"2024-04-23T18:36:02.765240+0000","last_undegraded":"2024-04-23T18:36:02.765240+0000","last_fullsized":"2024-04-23T18:36:02.765240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:06:20.645693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.725703+0000","last_change":"2024-04-23T18:34:01.233316+0000","last_active":"2024-04-23T18:36:02.725703+0000","last_peered":"2024-04-23T18:36:02.725703+0000","last_clean":"2024-04-23T18:36:02.725703+0000","last_became_active":"2024-04-23T18:34:01.231636+0000","last_became_peered":"2024-04-23T18:34:01.231636+0000","last_unstale":"2024-04-23T18:36:02.725703+0000","last_undegraded":"2024-04-23T18:36:02.725703+0000","last_fullsized":"2024-04-23T18:36:02.725703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:15.837129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765118+0000","last_change":"2024-04-23T18:34:01.228073+0000","last_active":"2024-04-23T18:36:02.765118+0000","last_peered":"2024-04-23T18:36:02.765118+0000","last_clean":"2024-04-23T18:36:02.765118+0000","last_became_active":"2024-04-23T18:34:01.227701+0000","last_became_peered":"2024-04-23T18:34:01.227701+0000","last_unstale":"2024-04-23T18:36:02.765118+0000","last_undegraded":"2024-04-23T18:36:02.765118+0000","last_fullsized":"2024-04-23T18:36:02.765118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:17:26.556498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732650+0000","last_change":"2024-04-23T18:34:55.258233+0000","last_active":"2024-04-23T18:36:02.732650+0000","last_peered":"2024-04-23T18:36:02.732650+0000","last_clean":"2024-04-23T18:36:02.732650+0000","last_became_active":"2024-04-23T18:34:55.257836+0000","last_became_peered":"2024-04-23T18:34:55.257836+0000","last_unstale":"2024-04-23T18:36:02.732650+0000","last_undegraded":"2024-04-23T18:36:02.732650+0000","last_fullsized":"2024-04-23T18:36:02.732650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:20.279369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765125+0000","last_change":"2024-04-23T18:34:55.260737+0000","last_active":"2024-04-23T18:36:02.765125+0000","last_peered":"2024-04-23T18:36:02.765125+0000","last_clean":"2024-04-23T18:36:02.765125+0000","last_became_active":"2024-04-23T18:34:55.260231+0000","last_became_peered":"2024-04-23T18:34:55.260231+0000","last_unstale":"2024-04-23T18:36:02.765125+0000","last_undegraded":"2024-04-23T18:36:02.765125+0000","last_fullsized":"2024-04-23T18:36:02.765125+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:45.292690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.721402+0000","last_change":"2024-04-23T18:34:55.259963+0000","last_active":"2024-04-23T18:36:02.721402+0000","last_peered":"2024-04-23T18:36:02.721402+0000","last_clean":"2024-04-23T18:36:02.721402+0000","last_became_active":"2024-04-23T18:34:55.259342+0000","last_became_peered":"2024-04-23T18:34:55.259342+0000","last_unstale":"2024-04-23T18:36:02.721402+0000","last_undegraded":"2024-04-23T18:36:02.721402+0000","last_fullsized":"2024-04-23T18:36:02.721402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:42:36.612809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765111+0000","last_change":"2024-04-23T18:34:01.234094+0000","last_active":"2024-04-23T18:36:02.765111+0000","last_peered":"2024-04-23T18:36:02.765111+0000","last_clean":"2024-04-23T18:36:02.765111+0000","last_became_active":"2024-04-23T18:34:01.232499+0000","last_became_peered":"2024-04-23T18:34:01.232499+0000","last_unstale":"2024-04-23T18:36:02.765111+0000","last_undegraded":"2024-04-23T18:36:02.765111+0000","last_fullsized":"2024-04-23T18:36:02.765111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:33.004245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732991+0000","last_change":"2024-04-23T18:34:01.229175+0000","last_active":"2024-04-23T18:36:02.732991+0000","last_peered":"2024-04-23T18:36:02.732991+0000","last_clean":"2024-04-23T18:36:02.732991+0000","last_became_active":"2024-04-23T18:34:01.228837+0000","last_became_peered":"2024-04-23T18:34:01.228837+0000","last_unstale":"2024-04-23T18:36:02.732991+0000","last_undegraded":"2024-04-23T18:36:02.732991+0000","last_fullsized":"2024-04-23T18:36:02.732991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:56:57.625110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765546+0000","last_change":"2024-04-23T18:34:01.235557+0000","last_active":"2024-04-23T18:36:02.765546+0000","last_peered":"2024-04-23T18:36:02.765546+0000","last_clean":"2024-04-23T18:36:02.765546+0000","last_became_active":"2024-04-23T18:34:01.235184+0000","last_became_peered":"2024-04-23T18:34:01.235184+0000","last_unstale":"2024-04-23T18:36:02.765546+0000","last_undegraded":"2024-04-23T18:36:02.765546+0000","last_fullsized":"2024-04-23T18:36:02.765546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:37:27.138041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.725500+0000","last_change":"2024-04-23T18:34:01.234070+0000","last_active":"2024-04-23T18:36:02.725500+0000","last_peered":"2024-04-23T18:36:02.725500+0000","last_clean":"2024-04-23T18:36:02.725500+0000","last_became_active":"2024-04-23T18:34:01.233727+0000","last_became_peered":"2024-04-23T18:34:01.233727+0000","last_unstale":"2024-04-23T18:36:02.725500+0000","last_undegraded":"2024-04-23T18:36:02.725500+0000","last_fullsized":"2024-04-23T18:36:02.725500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:59:16.489819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731502+0000","last_change":"2024-04-23T18:34:01.228256+0000","last_active":"2024-04-23T18:36:02.731502+0000","last_peered":"2024-04-23T18:36:02.731502+0000","last_clean":"2024-04-23T18:36:02.731502+0000","last_became_active":"2024-04-23T18:34:01.224400+0000","last_became_peered":"2024-04-23T18:34:01.224400+0000","last_unstale":"2024-04-23T18:36:02.731502+0000","last_undegraded":"2024-04-23T18:36:02.731502+0000","last_fullsized":"2024-04-23T18:36:02.731502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:37:28.401695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764642+0000","last_change":"2024-04-23T18:34:01.227250+0000","last_active":"2024-04-23T18:36:02.764642+0000","last_peered":"2024-04-23T18:36:02.764642+0000","last_clean":"2024-04-23T18:36:02.764642+0000","last_became_active":"2024-04-23T18:34:01.226937+0000","last_became_peered":"2024-04-23T18:34:01.226937+0000","last_unstale":"2024-04-23T18:36:02.764642+0000","last_undegraded":"2024-04-23T18:36:02.764642+0000","last_fullsized":"2024-04-23T18:36:02.764642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:26:20.801691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764582+0000","last_change":"2024-04-23T18:34:01.232351+0000","last_active":"2024-04-23T18:36:02.764582+0000","last_peered":"2024-04-23T18:36:02.764582+0000","last_clean":"2024-04-23T18:36:02.764582+0000","last_became_active":"2024-04-23T18:34:01.231987+0000","last_became_peered":"2024-04-23T18:34:01.231987+0000","last_unstale":"2024-04-23T18:36:02.764582+0000","last_undegraded":"2024-04-23T18:36:02.764582+0000","last_fullsized":"2024-04-23T18:36:02.764582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:50.886496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764465+0000","last_change":"2024-04-23T18:34:01.232196+0000","last_active":"2024-04-23T18:36:02.764465+0000","last_peered":"2024-04-23T18:36:02.764465+0000","last_clean":"2024-04-23T18:36:02.764465+0000","last_became_active":"2024-04-23T18:34:01.231746+0000","last_became_peered":"2024-04-23T18:34:01.231746+0000","last_unstale":"2024-04-23T18:36:02.764465+0000","last_undegraded":"2024-04-23T18:36:02.764465+0000","last_fullsized":"2024-04-23T18:36:02.764465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:58.888549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.735096+0000","last_change":"2024-04-23T18:34:01.231509+0000","last_active":"2024-04-23T18:36:02.735096+0000","last_peered":"2024-04-23T18:36:02.735096+0000","last_clean":"2024-04-23T18:36:02.735096+0000","last_became_active":"2024-04-23T18:34:01.231017+0000","last_became_peered":"2024-04-23T18:34:01.231017+0000","last_unstale":"2024-04-23T18:36:02.735096+0000","last_undegraded":"2024-04-23T18:36:02.735096+0000","last_fullsized":"2024-04-23T18:36:02.735096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:04:51.929462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.721702+0000","last_change":"2024-04-23T18:34:01.223285+0000","last_active":"2024-04-23T18:36:02.721702+0000","last_peered":"2024-04-23T18:36:02.721702+0000","last_clean":"2024-04-23T18:36:02.721702+0000","last_became_active":"2024-04-23T18:34:01.222962+0000","last_became_peered":"2024-04-23T18:34:01.222962+0000","last_unstale":"2024-04-23T18:36:02.721702+0000","last_undegraded":"2024-04-23T18:36:02.721702+0000","last_fullsized":"2024-04-23T18:36:02.721702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:18.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722773+0000","last_change":"2024-04-23T18:34:01.225922+0000","last_active":"2024-04-23T18:36:02.722773+0000","last_peered":"2024-04-23T18:36:02.722773+0000","last_clean":"2024-04-23T18:36:02.722773+0000","last_became_active":"2024-04-23T18:34:01.225235+0000","last_became_peered":"2024-04-23T18:34:01.225235+0000","last_unstale":"2024-04-23T18:36:02.722773+0000","last_undegraded":"2024-04-23T18:36:02.722773+0000","last_fullsized":"2024-04-23T18:36:02.722773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:33.480782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":163,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:07.705101+0000","last_change":"2024-04-23T18:34:00.224606+0000","last_active":"2024-04-23T18:36:07.705101+0000","last_peered":"2024-04-23T18:36:07.705101+0000","last_clean":"2024-04-23T18:36:07.705101+0000","last_became_active":"2024-04-23T18:34:00.224291+0000","last_became_peered":"2024-04-23T18:34:00.224291+0000","last_unstale":"2024-04-23T18:36:07.705101+0000","last_undegraded":"2024-04-23T18:36:07.705101+0000","last_fullsized":"2024-04-23T18:36:07.705101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:40.734704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765465+0000","last_change":"2024-04-23T18:34:01.235023+0000","last_active":"2024-04-23T18:36:02.765465+0000","last_peered":"2024-04-23T18:36:02.765465+0000","last_clean":"2024-04-23T18:36:02.765465+0000","last_became_active":"2024-04-23T18:34:01.234672+0000","last_became_peered":"2024-04-23T18:34:01.234672+0000","last_unstale":"2024-04-23T18:36:02.765465+0000","last_undegraded":"2024-04-23T18:36:02.765465+0000","last_fullsized":"2024-04-23T18:36:02.765465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:53:33.734343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765629+0000","last_change":"2024-04-23T18:34:00.232552+0000","last_active":"2024-04-23T18:36:02.765629+0000","last_peered":"2024-04-23T18:36:02.765629+0000","last_clean":"2024-04-23T18:36:02.765629+0000","last_became_active":"2024-04-23T18:34:00.232226+0000","last_became_peered":"2024-04-23T18:34:00.232226+0000","last_unstale":"2024-04-23T18:36:02.765629+0000","last_undegraded":"2024-04-23T18:36:02.765629+0000","last_fullsized":"2024-04-23T18:36:02.765629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.563757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765732+0000","last_change":"2024-04-23T18:34:01.236013+0000","last_active":"2024-04-23T18:36:02.765732+0000","last_peered":"2024-04-23T18:36:02.765732+0000","last_clean":"2024-04-23T18:36:02.765732+0000","last_became_active":"2024-04-23T18:34:01.235590+0000","last_became_peered":"2024-04-23T18:34:01.235590+0000","last_unstale":"2024-04-23T18:36:02.765732+0000","last_undegraded":"2024-04-23T18:36:02.765732+0000","last_fullsized":"2024-04-23T18:36:02.765732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:45.741648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'13","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765471+0000","last_change":"2024-04-23T18:34:55.298010+0000","last_active":"2024-04-23T18:36:02.765471+0000","last_peered":"2024-04-23T18:36:02.765471+0000","last_clean":"2024-04-23T18:36:02.765471+0000","last_became_active":"2024-04-23T18:34:55.288221+0000","last_became_peered":"2024-04-23T18:34:55.288221+0000","last_unstale":"2024-04-23T18:36:02.765471+0000","last_undegraded":"2024-04-23T18:36:02.765471+0000","last_fullsized":"2024-04-23T18:36:02.765471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:21:35.469676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764723+0000","last_change":"2024-04-23T18:34:01.230435+0000","last_active":"2024-04-23T18:36:02.764723+0000","last_peered":"2024-04-23T18:36:02.764723+0000","last_clean":"2024-04-23T18:36:02.764723+0000","last_became_active":"2024-04-23T18:34:01.230095+0000","last_became_peered":"2024-04-23T18:34:01.230095+0000","last_unstale":"2024-04-23T18:36:02.764723+0000","last_undegraded":"2024-04-23T18:36:02.764723+0000","last_fullsized":"2024-04-23T18:36:02.764723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:31.653254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765399+0000","last_change":"2024-04-23T18:34:00.227853+0000","last_active":"2024-04-23T18:36:02.765399+0000","last_peered":"2024-04-23T18:36:02.765399+0000","last_clean":"2024-04-23T18:36:02.765399+0000","last_became_active":"2024-04-23T18:34:00.226596+0000","last_became_peered":"2024-04-23T18:34:00.226596+0000","last_unstale":"2024-04-23T18:36:02.765399+0000","last_undegraded":"2024-04-23T18:36:02.765399+0000","last_fullsized":"2024-04-23T18:36:02.765399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:16:10.077969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765100+0000","last_change":"2024-04-23T18:34:01.235955+0000","last_active":"2024-04-23T18:36:02.765100+0000","last_peered":"2024-04-23T18:36:02.765100+0000","last_clean":"2024-04-23T18:36:02.765100+0000","last_became_active":"2024-04-23T18:34:01.235609+0000","last_became_peered":"2024-04-23T18:34:01.235609+0000","last_unstale":"2024-04-23T18:36:02.765100+0000","last_undegraded":"2024-04-23T18:36:02.765100+0000","last_fullsized":"2024-04-23T18:36:02.765100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:07:39.480213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":162,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:07.697050+0000","last_change":"2024-04-23T18:34:00.226319+0000","last_active":"2024-04-23T18:36:07.697050+0000","last_peered":"2024-04-23T18:36:07.697050+0000","last_clean":"2024-04-23T18:36:07.697050+0000","last_became_active":"2024-04-23T18:34:00.225796+0000","last_became_peered":"2024-04-23T18:34:00.225796+0000","last_unstale":"2024-04-23T18:36:07.697050+0000","last_undegraded":"2024-04-23T18:36:07.697050+0000","last_fullsized":"2024-04-23T18:36:07.697050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:11:18.309181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765238+0000","last_change":"2024-04-23T18:34:01.227336+0000","last_active":"2024-04-23T18:36:02.765238+0000","last_peered":"2024-04-23T18:36:02.765238+0000","last_clean":"2024-04-23T18:36:02.765238+0000","last_became_active":"2024-04-23T18:34:01.226987+0000","last_became_peered":"2024-04-23T18:34:01.226987+0000","last_unstale":"2024-04-23T18:36:02.765238+0000","last_undegraded":"2024-04-23T18:36:02.765238+0000","last_fullsized":"2024-04-23T18:36:02.765238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:22:30.058673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732492+0000","last_change":"2024-04-23T18:34:55.294322+0000","last_active":"2024-04-23T18:36:02.732492+0000","last_peered":"2024-04-23T18:36:02.732492+0000","last_clean":"2024-04-23T18:36:02.732492+0000","last_became_active":"2024-04-23T18:34:55.288855+0000","last_became_peered":"2024-04-23T18:34:55.288855+0000","last_unstale":"2024-04-23T18:36:02.732492+0000","last_undegraded":"2024-04-23T18:36:02.732492+0000","last_fullsized":"2024-04-23T18:36:02.732492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:07:50.097696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764793+0000","last_change":"2024-04-23T18:34:55.287339+0000","last_active":"2024-04-23T18:36:02.764793+0000","last_peered":"2024-04-23T18:36:02.764793+0000","last_clean":"2024-04-23T18:36:02.764793+0000","last_became_active":"2024-04-23T18:34:55.287062+0000","last_became_peered":"2024-04-23T18:34:55.287062+0000","last_unstale":"2024-04-23T18:36:02.764793+0000","last_undegraded":"2024-04-23T18:36:02.764793+0000","last_fullsized":"2024-04-23T18:36:02.764793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:44.864600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731488+0000","last_change":"2024-04-23T18:34:00.227595+0000","last_active":"2024-04-23T18:36:02.731488+0000","last_peered":"2024-04-23T18:36:02.731488+0000","last_clean":"2024-04-23T18:36:02.731488+0000","last_became_active":"2024-04-23T18:34:00.226830+0000","last_became_peered":"2024-04-23T18:34:00.226830+0000","last_unstale":"2024-04-23T18:36:02.731488+0000","last_undegraded":"2024-04-23T18:36:02.731488+0000","last_fullsized":"2024-04-23T18:36:02.731488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:20.661984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764758+0000","last_change":"2024-04-23T18:34:01.225053+0000","last_active":"2024-04-23T18:36:02.764758+0000","last_peered":"2024-04-23T18:36:02.764758+0000","last_clean":"2024-04-23T18:36:02.764758+0000","last_became_active":"2024-04-23T18:34:01.224723+0000","last_became_peered":"2024-04-23T18:34:01.224723+0000","last_unstale":"2024-04-23T18:36:02.764758+0000","last_undegraded":"2024-04-23T18:36:02.764758+0000","last_fullsized":"2024-04-23T18:36:02.764758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:01:51.486691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764506+0000","last_change":"2024-04-23T18:34:00.224783+0000","last_active":"2024-04-23T18:36:02.764506+0000","last_peered":"2024-04-23T18:36:02.764506+0000","last_clean":"2024-04-23T18:36:02.764506+0000","last_became_active":"2024-04-23T18:34:00.224436+0000","last_became_peered":"2024-04-23T18:34:00.224436+0000","last_unstale":"2024-04-23T18:36:02.764506+0000","last_undegraded":"2024-04-23T18:36:02.764506+0000","last_fullsized":"2024-04-23T18:36:02.764506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:33.829188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.721558+0000","last_change":"2024-04-23T18:34:01.224283+0000","last_active":"2024-04-23T18:36:02.721558+0000","last_peered":"2024-04-23T18:36:02.721558+0000","last_clean":"2024-04-23T18:36:02.721558+0000","last_became_active":"2024-04-23T18:34:01.223949+0000","last_became_peered":"2024-04-23T18:34:01.223949+0000","last_unstale":"2024-04-23T18:36:02.721558+0000","last_undegraded":"2024-04-23T18:36:02.721558+0000","last_fullsized":"2024-04-23T18:36:02.721558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:50.560687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":268,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.690843+0000","last_change":"2024-04-23T18:34:00.218453+0000","last_active":"2024-04-23T18:36:03.690843+0000","last_peered":"2024-04-23T18:36:03.690843+0000","last_clean":"2024-04-23T18:36:03.690843+0000","last_became_active":"2024-04-23T18:34:00.218036+0000","last_became_peered":"2024-04-23T18:34:00.218036+0000","last_unstale":"2024-04-23T18:36:03.690843+0000","last_undegraded":"2024-04-23T18:36:03.690843+0000","last_fullsized":"2024-04-23T18:36:03.690843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:49.485185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764932+0000","last_change":"2024-04-23T18:34:01.231143+0000","last_active":"2024-04-23T18:36:02.764932+0000","last_peered":"2024-04-23T18:36:02.764932+0000","last_clean":"2024-04-23T18:36:02.764932+0000","last_became_active":"2024-04-23T18:34:01.230812+0000","last_became_peered":"2024-04-23T18:34:01.230812+0000","last_unstale":"2024-04-23T18:36:02.764932+0000","last_undegraded":"2024-04-23T18:36:02.764932+0000","last_fullsized":"2024-04-23T18:36:02.764932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:28.704807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765595+0000","last_change":"2024-04-23T18:34:00.235711+0000","last_active":"2024-04-23T18:36:02.765595+0000","last_peered":"2024-04-23T18:36:02.765595+0000","last_clean":"2024-04-23T18:36:02.765595+0000","last_became_active":"2024-04-23T18:34:00.235333+0000","last_became_peered":"2024-04-23T18:34:00.235333+0000","last_unstale":"2024-04-23T18:36:02.765595+0000","last_undegraded":"2024-04-23T18:36:02.765595+0000","last_fullsized":"2024-04-23T18:36:02.765595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:51.320744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765599+0000","last_change":"2024-04-23T18:34:01.238291+0000","last_active":"2024-04-23T18:36:02.765599+0000","last_peered":"2024-04-23T18:36:02.765599+0000","last_clean":"2024-04-23T18:36:02.765599+0000","last_became_active":"2024-04-23T18:34:01.237972+0000","last_became_peered":"2024-04-23T18:34:01.237972+0000","last_unstale":"2024-04-23T18:36:02.765599+0000","last_undegraded":"2024-04-23T18:36:02.765599+0000","last_fullsized":"2024-04-23T18:36:02.765599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:36:17.108170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765280+0000","last_change":"2024-04-23T18:34:00.227249+0000","last_active":"2024-04-23T18:36:02.765280+0000","last_peered":"2024-04-23T18:36:02.765280+0000","last_clean":"2024-04-23T18:36:02.765280+0000","last_became_active":"2024-04-23T18:34:00.226854+0000","last_became_peered":"2024-04-23T18:34:00.226854+0000","last_unstale":"2024-04-23T18:36:02.765280+0000","last_undegraded":"2024-04-23T18:36:02.765280+0000","last_fullsized":"2024-04-23T18:36:02.765280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:49.619906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764813+0000","last_change":"2024-04-23T18:34:01.225518+0000","last_active":"2024-04-23T18:36:02.764813+0000","last_peered":"2024-04-23T18:36:02.764813+0000","last_clean":"2024-04-23T18:36:02.764813+0000","last_became_active":"2024-04-23T18:34:01.224907+0000","last_became_peered":"2024-04-23T18:34:01.224907+0000","last_unstale":"2024-04-23T18:36:02.764813+0000","last_undegraded":"2024-04-23T18:36:02.764813+0000","last_fullsized":"2024-04-23T18:36:02.764813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:17:19.876629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":307,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.690858+0000","last_change":"2024-04-23T18:34:00.224770+0000","last_active":"2024-04-23T18:36:03.690858+0000","last_peered":"2024-04-23T18:36:03.690858+0000","last_clean":"2024-04-23T18:36:03.690858+0000","last_became_active":"2024-04-23T18:34:00.224395+0000","last_became_peered":"2024-04-23T18:34:00.224395+0000","last_unstale":"2024-04-23T18:36:03.690858+0000","last_undegraded":"2024-04-23T18:36:03.690858+0000","last_fullsized":"2024-04-23T18:36:03.690858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:14:09.172479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60022,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":248,"num_read_kb":183,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765184+0000","last_change":"2024-04-23T18:34:01.227771+0000","last_active":"2024-04-23T18:36:02.765184+0000","last_peered":"2024-04-23T18:36:02.765184+0000","last_clean":"2024-04-23T18:36:02.765184+0000","last_became_active":"2024-04-23T18:34:01.225063+0000","last_became_peered":"2024-04-23T18:34:01.225063+0000","last_unstale":"2024-04-23T18:36:02.765184+0000","last_undegraded":"2024-04-23T18:36:02.765184+0000","last_fullsized":"2024-04-23T18:36:02.765184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:39:53.921587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722785+0000","last_change":"2024-04-23T18:34:00.230455+0000","last_active":"2024-04-23T18:36:02.722785+0000","last_peered":"2024-04-23T18:36:02.722785+0000","last_clean":"2024-04-23T18:36:02.722785+0000","last_became_active":"2024-04-23T18:34:00.229934+0000","last_became_peered":"2024-04-23T18:34:00.229934+0000","last_unstale":"2024-04-23T18:36:02.722785+0000","last_undegraded":"2024-04-23T18:36:02.722785+0000","last_fullsized":"2024-04-23T18:36:02.722785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:26:34.787606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731340+0000","last_change":"2024-04-23T18:34:01.225757+0000","last_active":"2024-04-23T18:36:02.731340+0000","last_peered":"2024-04-23T18:36:02.731340+0000","last_clean":"2024-04-23T18:36:02.731340+0000","last_became_active":"2024-04-23T18:34:01.225447+0000","last_became_peered":"2024-04-23T18:34:01.225447+0000","last_unstale":"2024-04-23T18:36:02.731340+0000","last_undegraded":"2024-04-23T18:36:02.731340+0000","last_fullsized":"2024-04-23T18:36:02.731340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:17.212135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764698+0000","last_change":"2024-04-23T18:34:00.231749+0000","last_active":"2024-04-23T18:36:02.764698+0000","last_peered":"2024-04-23T18:36:02.764698+0000","last_clean":"2024-04-23T18:36:02.764698+0000","last_became_active":"2024-04-23T18:34:00.231207+0000","last_became_peered":"2024-04-23T18:34:00.231207+0000","last_unstale":"2024-04-23T18:36:02.764698+0000","last_undegraded":"2024-04-23T18:36:02.764698+0000","last_fullsized":"2024-04-23T18:36:02.764698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:23:05.723265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765316+0000","last_change":"2024-04-23T18:35:55.690568+0000","last_active":"2024-04-23T18:36:02.765316+0000","last_peered":"2024-04-23T18:36:02.765316+0000","last_clean":"2024-04-23T18:36:02.765316+0000","last_became_active":"2024-04-23T18:35:55.690174+0000","last_became_peered":"2024-04-23T18:35:55.690174+0000","last_unstale":"2024-04-23T18:36:02.765316+0000","last_undegraded":"2024-04-23T18:36:02.765316+0000","last_fullsized":"2024-04-23T18:36:02.765316+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:29.462962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.734130+0000","last_change":"2024-04-23T18:34:00.228578+0000","last_active":"2024-04-23T18:36:03.734130+0000","last_peered":"2024-04-23T18:36:03.734130+0000","last_clean":"2024-04-23T18:36:03.734130+0000","last_became_active":"2024-04-23T18:34:00.228238+0000","last_became_peered":"2024-04-23T18:34:00.228238+0000","last_unstale":"2024-04-23T18:36:03.734130+0000","last_undegraded":"2024-04-23T18:36:03.734130+0000","last_fullsized":"2024-04-23T18:36:03.734130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:58:21.283593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764889+0000","last_change":"2024-04-23T18:34:01.239481+0000","last_active":"2024-04-23T18:36:02.764889+0000","last_peered":"2024-04-23T18:36:02.764889+0000","last_clean":"2024-04-23T18:36:02.764889+0000","last_became_active":"2024-04-23T18:34:01.239042+0000","last_became_peered":"2024-04-23T18:34:01.239042+0000","last_unstale":"2024-04-23T18:36:02.764889+0000","last_undegraded":"2024-04-23T18:36:02.764889+0000","last_fullsized":"2024-04-23T18:36:02.764889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:45.755462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734712+0000","last_change":"2024-04-23T18:34:55.273529+0000","last_active":"2024-04-23T18:36:02.734712+0000","last_peered":"2024-04-23T18:36:02.734712+0000","last_clean":"2024-04-23T18:36:02.734712+0000","last_became_active":"2024-04-23T18:34:55.259780+0000","last_became_peered":"2024-04-23T18:34:55.259780+0000","last_unstale":"2024-04-23T18:36:02.734712+0000","last_undegraded":"2024-04-23T18:36:02.734712+0000","last_fullsized":"2024-04-23T18:36:02.734712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:48:56.272606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764515+0000","last_change":"2024-04-23T18:34:01.239943+0000","last_active":"2024-04-23T18:36:02.764515+0000","last_peered":"2024-04-23T18:36:02.764515+0000","last_clean":"2024-04-23T18:36:02.764515+0000","last_became_active":"2024-04-23T18:34:01.239588+0000","last_became_peered":"2024-04-23T18:34:01.239588+0000","last_unstale":"2024-04-23T18:36:02.764515+0000","last_undegraded":"2024-04-23T18:36:02.764515+0000","last_fullsized":"2024-04-23T18:36:02.764515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:51:39.344341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765578+0000","last_change":"2024-04-23T18:34:00.227372+0000","last_active":"2024-04-23T18:36:02.765578+0000","last_peered":"2024-04-23T18:36:02.765578+0000","last_clean":"2024-04-23T18:36:02.765578+0000","last_became_active":"2024-04-23T18:34:00.226709+0000","last_became_peered":"2024-04-23T18:34:00.226709+0000","last_unstale":"2024-04-23T18:36:02.765578+0000","last_undegraded":"2024-04-23T18:36:02.765578+0000","last_fullsized":"2024-04-23T18:36:02.765578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:06:20.047108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734890+0000","last_change":"2024-04-23T18:34:01.235389+0000","last_active":"2024-04-23T18:36:02.734890+0000","last_peered":"2024-04-23T18:36:02.734890+0000","last_clean":"2024-04-23T18:36:02.734890+0000","last_became_active":"2024-04-23T18:34:01.235026+0000","last_became_peered":"2024-04-23T18:34:01.235026+0000","last_unstale":"2024-04-23T18:36:02.734890+0000","last_undegraded":"2024-04-23T18:36:02.734890+0000","last_fullsized":"2024-04-23T18:36:02.734890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:48.849525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.721668+0000","last_change":"2024-04-23T18:34:55.260852+0000","last_active":"2024-04-23T18:36:02.721668+0000","last_peered":"2024-04-23T18:36:02.721668+0000","last_clean":"2024-04-23T18:36:02.721668+0000","last_became_active":"2024-04-23T18:34:55.260452+0000","last_became_peered":"2024-04-23T18:34:55.260452+0000","last_unstale":"2024-04-23T18:36:02.721668+0000","last_undegraded":"2024-04-23T18:36:02.721668+0000","last_fullsized":"2024-04-23T18:36:02.721668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:28:13.496447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732819+0000","last_change":"2024-04-23T18:34:01.224992+0000","last_active":"2024-04-23T18:36:02.732819+0000","last_peered":"2024-04-23T18:36:02.732819+0000","last_clean":"2024-04-23T18:36:02.732819+0000","last_became_active":"2024-04-23T18:34:01.224508+0000","last_became_peered":"2024-04-23T18:34:01.224508+0000","last_unstale":"2024-04-23T18:36:02.732819+0000","last_undegraded":"2024-04-23T18:36:02.732819+0000","last_fullsized":"2024-04-23T18:36:02.732819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:57:24.183451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":152,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.700066+0000","last_change":"2024-04-23T18:34:00.234818+0000","last_active":"2024-04-23T18:36:03.700066+0000","last_peered":"2024-04-23T18:36:03.700066+0000","last_clean":"2024-04-23T18:36:03.700066+0000","last_became_active":"2024-04-23T18:34:00.234329+0000","last_became_peered":"2024-04-23T18:34:00.234329+0000","last_unstale":"2024-04-23T18:36:03.700066+0000","last_undegraded":"2024-04-23T18:36:03.700066+0000","last_fullsized":"2024-04-23T18:36:03.700066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:43:02.405694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.732868+0000","last_change":"2024-04-23T18:34:55.261073+0000","last_active":"2024-04-23T18:36:02.732868+0000","last_peered":"2024-04-23T18:36:02.732868+0000","last_clean":"2024-04-23T18:36:02.732868+0000","last_became_active":"2024-04-23T18:34:55.260655+0000","last_became_peered":"2024-04-23T18:34:55.260655+0000","last_unstale":"2024-04-23T18:36:02.732868+0000","last_undegraded":"2024-04-23T18:36:02.732868+0000","last_fullsized":"2024-04-23T18:36:02.732868+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:37:36.797785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764684+0000","last_change":"2024-04-23T18:34:01.234184+0000","last_active":"2024-04-23T18:36:02.764684+0000","last_peered":"2024-04-23T18:36:02.764684+0000","last_clean":"2024-04-23T18:36:02.764684+0000","last_became_active":"2024-04-23T18:34:01.233846+0000","last_became_peered":"2024-04-23T18:34:01.233846+0000","last_unstale":"2024-04-23T18:36:02.764684+0000","last_undegraded":"2024-04-23T18:36:02.764684+0000","last_fullsized":"2024-04-23T18:36:02.764684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:41.771495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:06.739034+0000","last_change":"2024-04-23T18:34:00.227862+0000","last_active":"2024-04-23T18:36:06.739034+0000","last_peered":"2024-04-23T18:36:06.739034+0000","last_clean":"2024-04-23T18:36:06.739034+0000","last_became_active":"2024-04-23T18:34:00.227515+0000","last_became_peered":"2024-04-23T18:34:00.227515+0000","last_unstale":"2024-04-23T18:36:06.739034+0000","last_undegraded":"2024-04-23T18:36:06.739034+0000","last_fullsized":"2024-04-23T18:36:06.739034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:18.595690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765322+0000","last_change":"2024-04-23T18:34:01.236338+0000","last_active":"2024-04-23T18:36:02.765322+0000","last_peered":"2024-04-23T18:36:02.765322+0000","last_clean":"2024-04-23T18:36:02.765322+0000","last_became_active":"2024-04-23T18:34:01.235795+0000","last_became_peered":"2024-04-23T18:34:01.235795+0000","last_unstale":"2024-04-23T18:36:02.765322+0000","last_undegraded":"2024-04-23T18:36:02.765322+0000","last_fullsized":"2024-04-23T18:36:02.765322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:45:01.928793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.725362+0000","last_change":"2024-04-23T18:34:00.221851+0000","last_active":"2024-04-23T18:36:02.725362+0000","last_peered":"2024-04-23T18:36:02.725362+0000","last_clean":"2024-04-23T18:36:02.725362+0000","last_became_active":"2024-04-23T18:34:00.221513+0000","last_became_peered":"2024-04-23T18:34:00.221513+0000","last_unstale":"2024-04-23T18:36:02.725362+0000","last_undegraded":"2024-04-23T18:36:02.725362+0000","last_fullsized":"2024-04-23T18:36:02.725362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:15:04.608759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764979+0000","last_change":"2024-04-23T18:32:48.218931+0000","last_active":"2024-04-23T18:36:02.764979+0000","last_peered":"2024-04-23T18:36:02.764979+0000","last_clean":"2024-04-23T18:36:02.764979+0000","last_became_active":"2024-04-23T18:32:48.218484+0000","last_became_peered":"2024-04-23T18:32:48.218484+0000","last_unstale":"2024-04-23T18:36:02.764979+0000","last_undegraded":"2024-04-23T18:36:02.764979+0000","last_fullsized":"2024-04-23T18:36:02.764979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:29:21.029319+0000","last_clean_scrub_stamp":"2024-04-23T18:29:21.029319+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:03:42.234165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731244+0000","last_change":"2024-04-23T18:34:01.220346+0000","last_active":"2024-04-23T18:36:02.731244+0000","last_peered":"2024-04-23T18:36:02.731244+0000","last_clean":"2024-04-23T18:36:02.731244+0000","last_became_active":"2024-04-23T18:34:01.219991+0000","last_became_peered":"2024-04-23T18:34:01.219991+0000","last_unstale":"2024-04-23T18:36:02.731244+0000","last_undegraded":"2024-04-23T18:36:02.731244+0000","last_fullsized":"2024-04-23T18:36:02.731244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:29:00.662700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765123+0000","last_change":"2024-04-23T18:34:00.229625+0000","last_active":"2024-04-23T18:36:02.765123+0000","last_peered":"2024-04-23T18:36:02.765123+0000","last_clean":"2024-04-23T18:36:02.765123+0000","last_became_active":"2024-04-23T18:34:00.229204+0000","last_became_peered":"2024-04-23T18:34:00.229204+0000","last_unstale":"2024-04-23T18:36:02.765123+0000","last_undegraded":"2024-04-23T18:36:02.765123+0000","last_fullsized":"2024-04-23T18:36:02.765123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:42.370870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765328+0000","last_change":"2024-04-23T18:34:00.236000+0000","last_active":"2024-04-23T18:36:02.765328+0000","last_peered":"2024-04-23T18:36:02.765328+0000","last_clean":"2024-04-23T18:36:02.765328+0000","last_became_active":"2024-04-23T18:34:00.235561+0000","last_became_peered":"2024-04-23T18:34:00.235561+0000","last_unstale":"2024-04-23T18:36:02.765328+0000","last_undegraded":"2024-04-23T18:36:02.765328+0000","last_fullsized":"2024-04-23T18:36:02.765328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:07.565489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731492+0000","last_change":"2024-04-23T18:34:01.227955+0000","last_active":"2024-04-23T18:36:02.731492+0000","last_peered":"2024-04-23T18:36:02.731492+0000","last_clean":"2024-04-23T18:36:02.731492+0000","last_became_active":"2024-04-23T18:34:01.225992+0000","last_became_peered":"2024-04-23T18:34:01.225992+0000","last_unstale":"2024-04-23T18:36:02.731492+0000","last_undegraded":"2024-04-23T18:36:02.731492+0000","last_fullsized":"2024-04-23T18:36:02.731492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:05:55.147008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'102","reported_seq":220,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:05.320009+0000","last_change":"2024-04-23T18:34:00.230133+0000","last_active":"2024-04-23T18:36:05.320009+0000","last_peered":"2024-04-23T18:36:05.320009+0000","last_clean":"2024-04-23T18:36:05.320009+0000","last_became_active":"2024-04-23T18:34:00.229596+0000","last_became_peered":"2024-04-23T18:34:00.229596+0000","last_unstale":"2024-04-23T18:36:05.320009+0000","last_undegraded":"2024-04-23T18:36:05.320009+0000","last_fullsized":"2024-04-23T18:36:05.320009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:23:06.064313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":185,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765529+0000","last_change":"2024-04-23T18:34:01.229066+0000","last_active":"2024-04-23T18:36:02.765529+0000","last_peered":"2024-04-23T18:36:02.765529+0000","last_clean":"2024-04-23T18:36:02.765529+0000","last_became_active":"2024-04-23T18:34:01.228711+0000","last_became_peered":"2024-04-23T18:34:01.228711+0000","last_unstale":"2024-04-23T18:36:02.765529+0000","last_undegraded":"2024-04-23T18:36:02.765529+0000","last_fullsized":"2024-04-23T18:36:02.765529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:58:30.601270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.725508+0000","last_change":"2024-04-23T18:34:00.231227+0000","last_active":"2024-04-23T18:36:02.725508+0000","last_peered":"2024-04-23T18:36:02.725508+0000","last_clean":"2024-04-23T18:36:02.725508+0000","last_became_active":"2024-04-23T18:34:00.230843+0000","last_became_peered":"2024-04-23T18:34:00.230843+0000","last_unstale":"2024-04-23T18:36:02.725508+0000","last_undegraded":"2024-04-23T18:36:02.725508+0000","last_fullsized":"2024-04-23T18:36:02.725508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:55.416591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765221+0000","last_change":"2024-04-23T18:34:01.233164+0000","last_active":"2024-04-23T18:36:02.765221+0000","last_peered":"2024-04-23T18:36:02.765221+0000","last_clean":"2024-04-23T18:36:02.765221+0000","last_became_active":"2024-04-23T18:34:01.232827+0000","last_became_peered":"2024-04-23T18:34:01.232827+0000","last_unstale":"2024-04-23T18:36:02.765221+0000","last_undegraded":"2024-04-23T18:36:02.765221+0000","last_fullsized":"2024-04-23T18:36:02.765221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:32:09.635515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764498+0000","last_change":"2024-04-23T18:34:00.225651+0000","last_active":"2024-04-23T18:36:02.764498+0000","last_peered":"2024-04-23T18:36:02.764498+0000","last_clean":"2024-04-23T18:36:02.764498+0000","last_became_active":"2024-04-23T18:34:00.225001+0000","last_became_peered":"2024-04-23T18:34:00.225001+0000","last_unstale":"2024-04-23T18:36:02.764498+0000","last_undegraded":"2024-04-23T18:36:02.764498+0000","last_fullsized":"2024-04-23T18:36:02.764498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:27:44.265388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765100+0000","last_change":"2024-04-23T18:34:01.224131+0000","last_active":"2024-04-23T18:36:02.765100+0000","last_peered":"2024-04-23T18:36:02.765100+0000","last_clean":"2024-04-23T18:36:02.765100+0000","last_became_active":"2024-04-23T18:34:01.223742+0000","last_became_peered":"2024-04-23T18:34:01.223742+0000","last_unstale":"2024-04-23T18:36:02.765100+0000","last_undegraded":"2024-04-23T18:36:02.765100+0000","last_fullsized":"2024-04-23T18:36:02.765100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:29:26.430964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764887+0000","last_change":"2024-04-23T18:34:00.235789+0000","last_active":"2024-04-23T18:36:02.764887+0000","last_peered":"2024-04-23T18:36:02.764887+0000","last_clean":"2024-04-23T18:36:02.764887+0000","last_became_active":"2024-04-23T18:34:00.235452+0000","last_became_peered":"2024-04-23T18:34:00.235452+0000","last_unstale":"2024-04-23T18:36:02.764887+0000","last_undegraded":"2024-04-23T18:36:02.764887+0000","last_fullsized":"2024-04-23T18:36:02.764887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:21.090121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734787+0000","last_change":"2024-04-23T18:34:01.226156+0000","last_active":"2024-04-23T18:36:02.734787+0000","last_peered":"2024-04-23T18:36:02.734787+0000","last_clean":"2024-04-23T18:36:02.734787+0000","last_became_active":"2024-04-23T18:34:01.225817+0000","last_became_peered":"2024-04-23T18:34:01.225817+0000","last_unstale":"2024-04-23T18:36:02.734787+0000","last_undegraded":"2024-04-23T18:36:02.734787+0000","last_fullsized":"2024-04-23T18:36:02.734787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:36.337139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.723101+0000","last_change":"2024-04-23T18:34:00.235116+0000","last_active":"2024-04-23T18:36:02.723101+0000","last_peered":"2024-04-23T18:36:02.723101+0000","last_clean":"2024-04-23T18:36:02.723101+0000","last_became_active":"2024-04-23T18:34:00.234521+0000","last_became_peered":"2024-04-23T18:34:00.234521+0000","last_unstale":"2024-04-23T18:36:02.723101+0000","last_undegraded":"2024-04-23T18:36:02.723101+0000","last_fullsized":"2024-04-23T18:36:02.723101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:51.061164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765111+0000","last_change":"2024-04-23T18:34:01.226445+0000","last_active":"2024-04-23T18:36:02.765111+0000","last_peered":"2024-04-23T18:36:02.765111+0000","last_clean":"2024-04-23T18:36:02.765111+0000","last_became_active":"2024-04-23T18:34:01.225875+0000","last_became_peered":"2024-04-23T18:34:01.225875+0000","last_unstale":"2024-04-23T18:36:02.765111+0000","last_undegraded":"2024-04-23T18:36:02.765111+0000","last_fullsized":"2024-04-23T18:36:02.765111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:48:38.712502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765246+0000","last_change":"2024-04-23T18:34:00.231918+0000","last_active":"2024-04-23T18:36:02.765246+0000","last_peered":"2024-04-23T18:36:02.765246+0000","last_clean":"2024-04-23T18:36:02.765246+0000","last_became_active":"2024-04-23T18:34:00.231550+0000","last_became_peered":"2024-04-23T18:34:00.231550+0000","last_unstale":"2024-04-23T18:36:02.765246+0000","last_undegraded":"2024-04-23T18:36:02.765246+0000","last_fullsized":"2024-04-23T18:36:02.765246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:52.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765279+0000","last_change":"2024-04-23T18:34:01.236360+0000","last_active":"2024-04-23T18:36:02.765279+0000","last_peered":"2024-04-23T18:36:02.765279+0000","last_clean":"2024-04-23T18:36:02.765279+0000","last_became_active":"2024-04-23T18:34:01.236056+0000","last_became_peered":"2024-04-23T18:34:01.236056+0000","last_unstale":"2024-04-23T18:36:02.765279+0000","last_undegraded":"2024-04-23T18:36:02.765279+0000","last_fullsized":"2024-04-23T18:36:02.765279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:54.535158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'9","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:03.736528+0000","last_change":"2024-04-23T18:34:00.233159+0000","last_active":"2024-04-23T18:36:03.736528+0000","last_peered":"2024-04-23T18:36:03.736528+0000","last_clean":"2024-04-23T18:36:03.736528+0000","last_became_active":"2024-04-23T18:34:00.232728+0000","last_became_peered":"2024-04-23T18:34:00.232728+0000","last_unstale":"2024-04-23T18:36:03.736528+0000","last_undegraded":"2024-04-23T18:36:03.736528+0000","last_fullsized":"2024-04-23T18:36:03.736528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:23.498057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2484,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765647+0000","last_change":"2024-04-23T18:34:01.226134+0000","last_active":"2024-04-23T18:36:02.765647+0000","last_peered":"2024-04-23T18:36:02.765647+0000","last_clean":"2024-04-23T18:36:02.765647+0000","last_became_active":"2024-04-23T18:34:01.225759+0000","last_became_peered":"2024-04-23T18:34:01.225759+0000","last_unstale":"2024-04-23T18:36:02.765647+0000","last_undegraded":"2024-04-23T18:36:02.765647+0000","last_fullsized":"2024-04-23T18:36:02.765647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:24.770595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'67","reported_seq":123,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:04.980284+0000","last_change":"2024-04-23T18:34:00.230054+0000","last_active":"2024-04-23T18:36:04.980284+0000","last_peered":"2024-04-23T18:36:04.980284+0000","last_clean":"2024-04-23T18:36:04.980284+0000","last_became_active":"2024-04-23T18:34:00.229621+0000","last_became_peered":"2024-04-23T18:34:00.229621+0000","last_unstale":"2024-04-23T18:36:04.980284+0000","last_undegraded":"2024-04-23T18:36:04.980284+0000","last_fullsized":"2024-04-23T18:36:04.980284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:13:59.317282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765433+0000","last_change":"2024-04-23T18:34:01.240053+0000","last_active":"2024-04-23T18:36:02.765433+0000","last_peered":"2024-04-23T18:36:02.765433+0000","last_clean":"2024-04-23T18:36:02.765433+0000","last_became_active":"2024-04-23T18:34:01.239716+0000","last_became_peered":"2024-04-23T18:34:01.239716+0000","last_unstale":"2024-04-23T18:36:02.765433+0000","last_undegraded":"2024-04-23T18:36:02.765433+0000","last_fullsized":"2024-04-23T18:36:02.765433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:54.874374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731781+0000","last_change":"2024-04-23T18:34:00.227914+0000","last_active":"2024-04-23T18:36:02.731781+0000","last_peered":"2024-04-23T18:36:02.731781+0000","last_clean":"2024-04-23T18:36:02.731781+0000","last_became_active":"2024-04-23T18:34:00.227540+0000","last_became_peered":"2024-04-23T18:34:00.227540+0000","last_unstale":"2024-04-23T18:36:02.731781+0000","last_undegraded":"2024-04-23T18:36:02.731781+0000","last_fullsized":"2024-04-23T18:36:02.731781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:33:59.193650+0000","last_clean_scrub_stamp":"2024-04-23T18:33:59.193650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:18.442397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765656+0000","last_change":"2024-04-23T18:34:01.227319+0000","last_active":"2024-04-23T18:36:02.765656+0000","last_peered":"2024-04-23T18:36:02.765656+0000","last_clean":"2024-04-23T18:36:02.765656+0000","last_became_active":"2024-04-23T18:34:01.226969+0000","last_became_peered":"2024-04-23T18:34:01.226969+0000","last_unstale":"2024-04-23T18:36:02.765656+0000","last_undegraded":"2024-04-23T18:36:02.765656+0000","last_fullsized":"2024-04-23T18:36:02.765656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:19.302388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734899+0000","last_change":"2024-04-23T18:34:01.234466+0000","last_active":"2024-04-23T18:36:02.734899+0000","last_peered":"2024-04-23T18:36:02.734899+0000","last_clean":"2024-04-23T18:36:02.734899+0000","last_became_active":"2024-04-23T18:34:01.234127+0000","last_became_peered":"2024-04-23T18:34:01.234127+0000","last_unstale":"2024-04-23T18:36:02.734899+0000","last_undegraded":"2024-04-23T18:36:02.734899+0000","last_fullsized":"2024-04-23T18:36:02.734899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:51:51.912320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722721+0000","last_change":"2024-04-23T18:34:01.229707+0000","last_active":"2024-04-23T18:36:02.722721+0000","last_peered":"2024-04-23T18:36:02.722721+0000","last_clean":"2024-04-23T18:36:02.722721+0000","last_became_active":"2024-04-23T18:34:01.229311+0000","last_became_peered":"2024-04-23T18:34:01.229311+0000","last_unstale":"2024-04-23T18:36:02.722721+0000","last_undegraded":"2024-04-23T18:36:02.722721+0000","last_fullsized":"2024-04-23T18:36:02.722721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:15:51.972436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722618+0000","last_change":"2024-04-23T18:34:01.227285+0000","last_active":"2024-04-23T18:36:02.722618+0000","last_peered":"2024-04-23T18:36:02.722618+0000","last_clean":"2024-04-23T18:36:02.722618+0000","last_became_active":"2024-04-23T18:34:01.226960+0000","last_became_peered":"2024-04-23T18:34:01.226960+0000","last_unstale":"2024-04-23T18:36:02.722618+0000","last_undegraded":"2024-04-23T18:36:02.722618+0000","last_fullsized":"2024-04-23T18:36:02.722618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:50.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765008+0000","last_change":"2024-04-23T18:34:01.230697+0000","last_active":"2024-04-23T18:36:02.765008+0000","last_peered":"2024-04-23T18:36:02.765008+0000","last_clean":"2024-04-23T18:36:02.765008+0000","last_became_active":"2024-04-23T18:34:01.230352+0000","last_became_peered":"2024-04-23T18:34:01.230352+0000","last_unstale":"2024-04-23T18:36:02.765008+0000","last_undegraded":"2024-04-23T18:36:02.765008+0000","last_fullsized":"2024-04-23T18:36:02.765008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:35:10.609773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731645+0000","last_change":"2024-04-23T18:34:01.226970+0000","last_active":"2024-04-23T18:36:02.731645+0000","last_peered":"2024-04-23T18:36:02.731645+0000","last_clean":"2024-04-23T18:36:02.731645+0000","last_became_active":"2024-04-23T18:34:01.224151+0000","last_became_peered":"2024-04-23T18:34:01.224151+0000","last_unstale":"2024-04-23T18:36:02.731645+0000","last_undegraded":"2024-04-23T18:36:02.731645+0000","last_fullsized":"2024-04-23T18:36:02.731645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:36:15.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734930+0000","last_change":"2024-04-23T18:34:01.234194+0000","last_active":"2024-04-23T18:36:02.734930+0000","last_peered":"2024-04-23T18:36:02.734930+0000","last_clean":"2024-04-23T18:36:02.734930+0000","last_became_active":"2024-04-23T18:34:01.232404+0000","last_became_peered":"2024-04-23T18:34:01.232404+0000","last_unstale":"2024-04-23T18:36:02.734930+0000","last_undegraded":"2024-04-23T18:36:02.734930+0000","last_fullsized":"2024-04-23T18:36:02.734930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:30.691223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.722974+0000","last_change":"2024-04-23T18:34:01.219507+0000","last_active":"2024-04-23T18:36:02.722974+0000","last_peered":"2024-04-23T18:36:02.722974+0000","last_clean":"2024-04-23T18:36:02.722974+0000","last_became_active":"2024-04-23T18:34:01.218997+0000","last_became_peered":"2024-04-23T18:34:01.218997+0000","last_unstale":"2024-04-23T18:36:02.722974+0000","last_undegraded":"2024-04-23T18:36:02.722974+0000","last_fullsized":"2024-04-23T18:36:02.722974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:38.233652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.765439+0000","last_change":"2024-04-23T18:34:01.231846+0000","last_active":"2024-04-23T18:36:02.765439+0000","last_peered":"2024-04-23T18:36:02.765439+0000","last_clean":"2024-04-23T18:36:02.765439+0000","last_became_active":"2024-04-23T18:34:01.231467+0000","last_became_peered":"2024-04-23T18:34:01.231467+0000","last_unstale":"2024-04-23T18:36:02.765439+0000","last_undegraded":"2024-04-23T18:36:02.765439+0000","last_fullsized":"2024-04-23T18:36:02.765439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:37:27.891180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764528+0000","last_change":"2024-04-23T18:34:01.237086+0000","last_active":"2024-04-23T18:36:02.764528+0000","last_peered":"2024-04-23T18:36:02.764528+0000","last_clean":"2024-04-23T18:36:02.764528+0000","last_became_active":"2024-04-23T18:34:01.236610+0000","last_became_peered":"2024-04-23T18:34:01.236610+0000","last_unstale":"2024-04-23T18:36:02.764528+0000","last_undegraded":"2024-04-23T18:36:02.764528+0000","last_fullsized":"2024-04-23T18:36:02.764528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:24.916553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764832+0000","last_change":"2024-04-23T18:34:01.231091+0000","last_active":"2024-04-23T18:36:02.764832+0000","last_peered":"2024-04-23T18:36:02.764832+0000","last_clean":"2024-04-23T18:36:02.764832+0000","last_became_active":"2024-04-23T18:34:01.230731+0000","last_became_peered":"2024-04-23T18:34:01.230731+0000","last_unstale":"2024-04-23T18:36:02.764832+0000","last_undegraded":"2024-04-23T18:36:02.764832+0000","last_fullsized":"2024-04-23T18:36:02.764832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:22.771515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.764631+0000","last_change":"2024-04-23T18:34:01.231820+0000","last_active":"2024-04-23T18:36:02.764631+0000","last_peered":"2024-04-23T18:36:02.764631+0000","last_clean":"2024-04-23T18:36:02.764631+0000","last_became_active":"2024-04-23T18:34:01.230320+0000","last_became_peered":"2024-04-23T18:34:01.230320+0000","last_unstale":"2024-04-23T18:36:02.764631+0000","last_undegraded":"2024-04-23T18:36:02.764631+0000","last_fullsized":"2024-04-23T18:36:02.764631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:27:35.324323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.731327+0000","last_change":"2024-04-23T18:34:01.228723+0000","last_active":"2024-04-23T18:36:02.731327+0000","last_peered":"2024-04-23T18:36:02.731327+0000","last_clean":"2024-04-23T18:36:02.731327+0000","last_became_active":"2024-04-23T18:34:01.227500+0000","last_became_peered":"2024-04-23T18:34:01.227500+0000","last_unstale":"2024-04-23T18:36:02.731327+0000","last_undegraded":"2024-04-23T18:36:02.731327+0000","last_fullsized":"2024-04-23T18:36:02.731327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:17.908021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:36:02.734867+0000","last_change":"2024-04-23T18:34:01.227541+0000","last_active":"2024-04-23T18:36:02.734867+0000","last_peered":"2024-04-23T18:36:02.734867+0000","last_clean":"2024-04-23T18:36:02.734867+0000","last_became_active":"2024-04-23T18:34:01.227201+0000","last_became_peered":"2024-04-23T18:34:01.227201+0000","last_unstale":"2024-04-23T18:36:02.734867+0000","last_undegraded":"2024-04-23T18:36:02.734867+0000","last_fullsized":"2024-04-23T18:36:02.734867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:34:00.202752+0000","last_clean_scrub_stamp":"2024-04-23T18:34:00.202752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:25:24.886850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107658,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":791219,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1588,"num_read_kb":1010,"num_write":500,"num_write_kb":2375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":737280,"data_stored":2176634,"data_compressed":99906,"data_compressed_allocated":135168,"data_compressed_original":1941504,"omap_allocated":0,"internal_metadata":0},"log_size":380,"ondisk_log_size":380,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":488912,"data_compressed":14490,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1171456,"data_stored":1271310,"data_compressed":58406,"data_compressed_allocated":77824,"data_compressed_original":761856,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":881414,"data_compressed":33779,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037813,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":881680,"data_compressed":33636,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":1081543,"data_compressed":50376,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":489005,"data_compressed":14490,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":549158,"data_compressed":16364,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":490568,"data_compressed":14490,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":550134,"data_compressed":16364,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":1209188,"data_compressed":48981,"data_compressed_allocated":65536,"data_compressed_original":741376,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":488167,"data_compressed":14490,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1409536,"data_compressed":65578,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35886,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62473,"data_compressed":1874,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63840,"data_compressed":1874,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331975,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":193749,"data_compressed":8030,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:36:08.980 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:36:09.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:08 smithi157 ceph-mon[32580]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:36:09.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:08 smithi157 ceph-mon[32580]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:36:09.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:08 smithi157 ceph-mon[32580]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:36:09.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:08 smithi157 ceph-mon[32580]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:36:09.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:08 smithi157 ceph-mon[32580]: Updating smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:36:09.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:08 smithi157 ceph-mon[32580]: Updating smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:36:09.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:08 smithi157 ceph-mon[32580]: Updating smithi132:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:36:09.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:08 smithi157 ceph-mon[32580]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-23T18:36:09.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:08 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/28674201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:36:09.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:08 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/777435506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:36:09.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.122+0000 7feb11930640 1 -- 172.21.15.132:0/3523618122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb0c101ac0 msgr2=0x7feb0c105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.122+0000 7feb11930640 1 --2- 172.21.15.132:0/3523618122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb0c101ac0 0x7feb0c105ab0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7feafc009900 tx=0x7feafc02a410 comp rx=0 tx=0).stop 2024-04-23T18:36:09.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.123+0000 7feb11930640 1 -- 172.21.15.132:0/3523618122 shutdown_connections 2024-04-23T18:36:09.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.123+0000 7feb11930640 1 --2- 172.21.15.132:0/3523618122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb0c106190 0x7feb0c10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.123+0000 7feb11930640 1 --2- 172.21.15.132:0/3523618122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb0c101ac0 0x7feb0c105ab0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.123+0000 7feb11930640 1 --2- 172.21.15.132:0/3523618122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0c101110 0x7feb0c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.123+0000 7feb11930640 1 -- 172.21.15.132:0/3523618122 >> 172.21.15.132:0/3523618122 conn(0x7feb0c077fb0 msgr2=0x7feb0c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:09.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.123+0000 7feb11930640 1 -- 172.21.15.132:0/3523618122 shutdown_connections 2024-04-23T18:36:09.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.123+0000 7feb11930640 1 -- 172.21.15.132:0/3523618122 wait complete. 2024-04-23T18:36:09.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.123+0000 7feb11930640 1 Processor -- start 2024-04-23T18:36:09.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.123+0000 7feb11930640 1 -- start start 2024-04-23T18:36:09.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.123+0000 7feb11930640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb0c101110 0x7feb0c197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:09.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb11930640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb0c106190 0x7feb0c197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:09.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb11930640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0c19bf30 0x7feb0c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:09.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb11930640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1110d0 con 0x7feb0c101110 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb11930640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c110f50 con 0x7feb0c106190 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb11930640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c111250 con 0x7feb0c19bf30 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb0affd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb0c101110 0x7feb0c197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb0affd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb0c101110 0x7feb0c197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57856/0 (socket says 172.21.15.132:57856) 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb0affd640 1 -- 172.21.15.132:0/2350752694 learned_addr learned my addr 172.21.15.132:0/2350752694 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb0a7fc640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb0c106190 0x7feb0c197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb0b7fe640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0c19bf30 0x7feb0c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb0affd640 1 -- 172.21.15.132:0/2350752694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0c19bf30 msgr2=0x7feb0c19c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb0affd640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0c19bf30 0x7feb0c19c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb0affd640 1 -- 172.21.15.132:0/2350752694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb0c106190 msgr2=0x7feb0c197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb0affd640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb0c106190 0x7feb0c197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb0affd640 1 -- 172.21.15.132:0/2350752694 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c19ca20 con 0x7feb0c101110 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb0affd640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb0c101110 0x7feb0c197670 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feb0000d770 tx=0x7feb0000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb1092e640 1 -- 172.21.15.132:0/2350752694 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00017070 con 0x7feb0c101110 2024-04-23T18:36:09.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb1092e640 1 -- 172.21.15.132:0/2350752694 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb000054d0 con 0x7feb0c101110 2024-04-23T18:36:09.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb11930640 1 -- 172.21.15.132:0/2350752694 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0c19cd10 con 0x7feb0c101110 2024-04-23T18:36:09.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb1092e640 1 -- 172.21.15.132:0/2350752694 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00004c30 con 0x7feb0c101110 2024-04-23T18:36:09.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.124+0000 7feb11930640 1 -- 172.21.15.132:0/2350752694 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0c1a44d0 con 0x7feb0c101110 2024-04-23T18:36:09.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.125+0000 7feb11930640 1 -- 172.21.15.132:0/2350752694 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0c0691c0 con 0x7feb0c101110 2024-04-23T18:36:09.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.126+0000 7feb1092e640 1 -- 172.21.15.132:0/2350752694 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7feb00004dd0 con 0x7feb0c101110 2024-04-23T18:36:09.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.127+0000 7feb1092e640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7feaf40a3ec0 0x7feaf40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:09.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.127+0000 7feb0a7fc640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7feaf40a3ec0 0x7feaf40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:09.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.127+0000 7feb1092e640 1 -- 172.21.15.132:0/2350752694 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7feb000d0940 con 0x7feb0c101110 2024-04-23T18:36:09.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.127+0000 7feb0a7fc640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7feaf40a3ec0 0x7feaf40a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feafc000c00 tx=0x7feafc031400 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:09.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.130+0000 7feb1092e640 1 -- 172.21.15.132:0/2350752694 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb0009ce00 con 0x7feb0c101110 2024-04-23T18:36:09.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.347+0000 7feb11930640 1 -- 172.21.15.132:0/2350752694 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb0c1989c0 con 0x7feb0c101110 2024-04-23T18:36:09.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.348+0000 7feb1092e640 1 -- 172.21.15.132:0/2350752694 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11946 (secure 0 0 0) 0x7feb0c1989c0 con 0x7feb0c101110 2024-04-23T18:36:09.340 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:09.340 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:36:01.455959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24460,24425,24394,24491,24515]},"id":1}]} 2024-04-23T18:36:09.341 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 26 2024-04-23T18:36:09.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.350+0000 7feb11930640 1 -- 172.21.15.132:0/2350752694 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7feaf40a3ec0 msgr2=0x7feaf40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.350+0000 7feb11930640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7feaf40a3ec0 0x7feaf40a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feafc000c00 tx=0x7feafc031400 comp rx=0 tx=0).stop 2024-04-23T18:36:09.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.350+0000 7feb11930640 1 -- 172.21.15.132:0/2350752694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb0c101110 msgr2=0x7feb0c197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.350+0000 7feb11930640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb0c101110 0x7feb0c197670 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feb0000d770 tx=0x7feb0000dc30 comp rx=0 tx=0).stop 2024-04-23T18:36:09.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.350+0000 7feb11930640 1 -- 172.21.15.132:0/2350752694 shutdown_connections 2024-04-23T18:36:09.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.350+0000 7feb11930640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7feaf40a3ec0 0x7feaf40a6380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.350+0000 7feb11930640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0c19bf30 0x7feb0c19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.350+0000 7feb11930640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb0c106190 0x7feb0c197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.350+0000 7feb11930640 1 --2- 172.21.15.132:0/2350752694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb0c101110 0x7feb0c197670 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.350+0000 7feb11930640 1 -- 172.21.15.132:0/2350752694 >> 172.21.15.132:0/2350752694 conn(0x7feb0c077fb0 msgr2=0x7feb0c0ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:09.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.350+0000 7feb11930640 1 -- 172.21.15.132:0/2350752694 shutdown_connections 2024-04-23T18:36:09.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.350+0000 7feb11930640 1 -- 172.21.15.132:0/2350752694 wait complete. 2024-04-23T18:36:09.354 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:36:09.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.461+0000 7f195e55d640 1 -- 172.21.15.132:0/1558496277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958101110 msgr2=0x7f19581014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.461+0000 7f195e55d640 1 --2- 172.21.15.132:0/1558496277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958101110 0x7f19581014f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1948009900 tx=0x7f194802a410 comp rx=0 tx=0).stop 2024-04-23T18:36:09.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.461+0000 7f195e55d640 1 -- 172.21.15.132:0/1558496277 shutdown_connections 2024-04-23T18:36:09.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.461+0000 7f195e55d640 1 --2- 172.21.15.132:0/1558496277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1958106190 0x7f195810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.461+0000 7f195e55d640 1 --2- 172.21.15.132:0/1558496277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1958101ac0 0x7f1958105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.461+0000 7f195e55d640 1 --2- 172.21.15.132:0/1558496277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958101110 0x7f19581014f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.461+0000 7f195e55d640 1 -- 172.21.15.132:0/1558496277 >> 172.21.15.132:0/1558496277 conn(0x7f1958077fb0 msgr2=0x7f19580783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:09.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.461+0000 7f195e55d640 1 -- 172.21.15.132:0/1558496277 shutdown_connections 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.461+0000 7f195e55d640 1 -- 172.21.15.132:0/1558496277 wait complete. 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.461+0000 7f195e55d640 1 Processor -- start 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.461+0000 7f195e55d640 1 -- start start 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195e55d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1958101110 0x7f195819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195e55d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1958106190 0x7f195819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195e55d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958198680 0x7f1958198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195e55d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1958112210 con 0x7f1958198680 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195e55d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1958112090 con 0x7f1958106190 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195e55d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1958112390 con 0x7f1958101110 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195cafa640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958198680 0x7f1958198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195cafa640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958198680 0x7f1958198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58504/0 (socket says 172.21.15.132:58504) 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195cafa640 1 -- 172.21.15.132:0/2696227550 learned_addr learned my addr 172.21.15.132:0/2696227550 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f19577fe640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1958106190 0x7f195819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:09.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f1957fff640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1958101110 0x7f195819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:09.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195cafa640 1 -- 172.21.15.132:0/2696227550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1958101110 msgr2=0x7f195819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195cafa640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1958101110 0x7f195819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195cafa640 1 -- 172.21.15.132:0/2696227550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1958106190 msgr2=0x7f195819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195cafa640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1958106190 0x7f195819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195cafa640 1 -- 172.21.15.132:0/2696227550 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1958199280 con 0x7f1958198680 2024-04-23T18:36:09.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195cafa640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958198680 0x7f1958198b00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f195000da40 tx=0x7f195000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:09.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f19557fa640 1 -- 172.21.15.132:0/2696227550 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950017070 con 0x7f1958198680 2024-04-23T18:36:09.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.462+0000 7f195e55d640 1 -- 172.21.15.132:0/2696227550 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19581a5310 con 0x7f1958198680 2024-04-23T18:36:09.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.463+0000 7f19557fa640 1 -- 172.21.15.132:0/2696227550 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1950003e20 con 0x7f1958198680 2024-04-23T18:36:09.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.463+0000 7f19557fa640 1 -- 172.21.15.132:0/2696227550 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950010460 con 0x7f1958198680 2024-04-23T18:36:09.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.463+0000 7f195e55d640 1 -- 172.21.15.132:0/2696227550 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19581a5710 con 0x7f1958198680 2024-04-23T18:36:09.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.464+0000 7f195e55d640 1 -- 172.21.15.132:0/2696227550 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1958102590 con 0x7f1958198680 2024-04-23T18:36:09.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.465+0000 7f19557fa640 1 -- 172.21.15.132:0/2696227550 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f1950004020 con 0x7f1958198680 2024-04-23T18:36:09.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.465+0000 7f19557fa640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f19400a3e40 0x7f19400a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:09.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.465+0000 7f1957fff640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f19400a3e40 0x7f19400a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:09.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.465+0000 7f19557fa640 1 -- 172.21.15.132:0/2696227550 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f19500d0680 con 0x7f1958198680 2024-04-23T18:36:09.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.466+0000 7f1957fff640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f19400a3e40 0x7f19400a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1948031440 tx=0x7f1948005b50 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:09.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.470+0000 7f19557fa640 1 -- 172.21.15.132:0/2696227550 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f195009c470 con 0x7f1958198680 2024-04-23T18:36:09.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:09 smithi132 ceph-mon[29178]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:36:09.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:09 smithi132 ceph-mon[29178]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:36:09.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:09 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2350752694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:09.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.653+0000 7f195e55d640 1 -- 172.21.15.132:0/2696227550 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19581093c0 con 0x7f1958198680 2024-04-23T18:36:09.647 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.655+0000 7f19557fa640 1 -- 172.21.15.132:0/2696227550 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23228 (secure 0 0 0) 0x7f19500a0770 con 0x7f1958198680 2024-04-23T18:36:09.647 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:09.648 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":91,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:36:02.717304+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/2124409853":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/3095429281":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/899407689":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1857586348":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:36:09.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.657+0000 7f195e55d640 1 -- 172.21.15.132:0/2696227550 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f19400a3e40 msgr2=0x7f19400a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.657+0000 7f195e55d640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f19400a3e40 0x7f19400a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1948031440 tx=0x7f1948005b50 comp rx=0 tx=0).stop 2024-04-23T18:36:09.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.657+0000 7f195e55d640 1 -- 172.21.15.132:0/2696227550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958198680 msgr2=0x7f1958198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.657+0000 7f195e55d640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958198680 0x7f1958198b00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f195000da40 tx=0x7f195000df00 comp rx=0 tx=0).stop 2024-04-23T18:36:09.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.657+0000 7f195e55d640 1 -- 172.21.15.132:0/2696227550 shutdown_connections 2024-04-23T18:36:09.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.657+0000 7f195e55d640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f19400a3e40 0x7f19400a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.657+0000 7f195e55d640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958198680 0x7f1958198b00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.657+0000 7f195e55d640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1958106190 0x7f195819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.657+0000 7f195e55d640 1 --2- 172.21.15.132:0/2696227550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1958101110 0x7f195819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.657+0000 7f195e55d640 1 -- 172.21.15.132:0/2696227550 >> 172.21.15.132:0/2696227550 conn(0x7f1958077fb0 msgr2=0x7f19580ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:09.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.657+0000 7f195e55d640 1 -- 172.21.15.132:0/2696227550 shutdown_connections 2024-04-23T18:36:09.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.657+0000 7f195e55d640 1 -- 172.21.15.132:0/2696227550 wait complete. 2024-04-23T18:36:09.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:29:20.132451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:33:58.407521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:33:59.518206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:36:09.662 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:36:09.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.771+0000 7fa77c491640 1 -- 172.21.15.132:0/1207012086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77410c510 msgr2=0x7fa77410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.771+0000 7fa77c491640 1 --2- 172.21.15.132:0/1207012086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77410c510 0x7fa77410e9b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009ca0 tx=0x7fa76c02a9e0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.771+0000 7fa77c491640 1 -- 172.21.15.132:0/1207012086 shutdown_connections 2024-04-23T18:36:09.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.771+0000 7fa77c491640 1 --2- 172.21.15.132:0/1207012086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77410c510 0x7fa77410e9b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.771+0000 7fa77c491640 1 --2- 172.21.15.132:0/1207012086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa774103af0 0x7fa77410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.771+0000 7fa77c491640 1 --2- 172.21.15.132:0/1207012086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7741031d0 0x7fa7741035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.771+0000 7fa77c491640 1 -- 172.21.15.132:0/1207012086 >> 172.21.15.132:0/1207012086 conn(0x7fa774077f50 msgr2=0x7fa774078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:09.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.771+0000 7fa77c491640 1 -- 172.21.15.132:0/1207012086 shutdown_connections 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.771+0000 7fa77c491640 1 -- 172.21.15.132:0/1207012086 wait complete. 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa77c491640 1 Processor -- start 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa77c491640 1 -- start start 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa77c491640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7741031d0 0x7fa774198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa77c491640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa774103af0 0x7fa774198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa77c491640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77410c510 0x7fa77419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa77c491640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa7741121a0 con 0x7fa774103af0 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa77c491640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa774112020 con 0x7fa7741031d0 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa77c491640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa774112320 con 0x7fa77410c510 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa779a2c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa774103af0 0x7fa774198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa779a2c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa774103af0 0x7fa774198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58520/0 (socket says 172.21.15.132:58520) 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa779a2c640 1 -- 172.21.15.132:0/1590289626 learned_addr learned my addr 172.21.15.132:0/1590289626 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa77a22d640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7741031d0 0x7fa774198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.772+0000 7fa77aa2e640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77410c510 0x7fa77419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:09.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.773+0000 7fa779a2c640 1 -- 172.21.15.132:0/1590289626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77410c510 msgr2=0x7fa77419cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.773+0000 7fa779a2c640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77410c510 0x7fa77419cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.773+0000 7fa779a2c640 1 -- 172.21.15.132:0/1590289626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7741031d0 msgr2=0x7fa774198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.773+0000 7fa779a2c640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7741031d0 0x7fa774198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.773+0000 7fa779a2c640 1 -- 172.21.15.132:0/1590289626 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77419d6f0 con 0x7fa774103af0 2024-04-23T18:36:09.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.773+0000 7fa779a2c640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa774103af0 0x7fa774198c40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa76400e9f0 tx=0x7fa76400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:09.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.773+0000 7fa76b7fe640 1 -- 172.21.15.132:0/1590289626 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76400cd80 con 0x7fa774103af0 2024-04-23T18:36:09.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.773+0000 7fa76b7fe640 1 -- 172.21.15.132:0/1590289626 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7640108e0 con 0x7fa774103af0 2024-04-23T18:36:09.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.773+0000 7fa77c491640 1 -- 172.21.15.132:0/1590289626 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77419d9e0 con 0x7fa774103af0 2024-04-23T18:36:09.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.773+0000 7fa76b7fe640 1 -- 172.21.15.132:0/1590289626 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764018980 con 0x7fa774103af0 2024-04-23T18:36:09.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.773+0000 7fa77c491640 1 -- 172.21.15.132:0/1590289626 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7741a5270 con 0x7fa774103af0 2024-04-23T18:36:09.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.774+0000 7fa77c491640 1 -- 172.21.15.132:0/1590289626 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa774068d70 con 0x7fa774103af0 2024-04-23T18:36:09.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.775+0000 7fa76b7fe640 1 -- 172.21.15.132:0/1590289626 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fa764010430 con 0x7fa774103af0 2024-04-23T18:36:09.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.776+0000 7fa76b7fe640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa7600a3ec0 0x7fa7600a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:09.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.776+0000 7fa77a22d640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa7600a3ec0 0x7fa7600a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:09.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.776+0000 7fa76b7fe640 1 -- 172.21.15.132:0/1590289626 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fa7640d11e0 con 0x7fa774103af0 2024-04-23T18:36:09.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.776+0000 7fa77a22d640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa7600a3ec0 0x7fa7600a6380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa770005fd0 tx=0x7fa77000b390 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:09.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.780+0000 7fa76b7fe640 1 -- 172.21.15.132:0/1590289626 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa76409d620 con 0x7fa774103af0 2024-04-23T18:36:09.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.961+0000 7fa77c491640 1 -- 172.21.15.132:0/1590289626 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa774107100 con 0x7fa774103af0 2024-04-23T18:36:09.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.962+0000 7fa76b7fe640 1 -- 172.21.15.132:0/1590289626 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fa76409cf20 con 0x7fa774103af0 2024-04-23T18:36:09.954 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-04-23T18:36:09.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.964+0000 7fa77c491640 1 -- 172.21.15.132:0/1590289626 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa7600a3ec0 msgr2=0x7fa7600a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.964+0000 7fa77c491640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa7600a3ec0 0x7fa7600a6380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa770005fd0 tx=0x7fa77000b390 comp rx=0 tx=0).stop 2024-04-23T18:36:09.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.964+0000 7fa77c491640 1 -- 172.21.15.132:0/1590289626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa774103af0 msgr2=0x7fa774198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:09.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.964+0000 7fa77c491640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa774103af0 0x7fa774198c40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa76400e9f0 tx=0x7fa76400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.964+0000 7fa77c491640 1 -- 172.21.15.132:0/1590289626 shutdown_connections 2024-04-23T18:36:09.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.964+0000 7fa77c491640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa7600a3ec0 0x7fa7600a6380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.964+0000 7fa77c491640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa77410c510 0x7fa77419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.964+0000 7fa77c491640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa774103af0 0x7fa774198c40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.964+0000 7fa77c491640 1 --2- 172.21.15.132:0/1590289626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7741031d0 0x7fa774198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:09.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.964+0000 7fa77c491640 1 -- 172.21.15.132:0/1590289626 >> 172.21.15.132:0/1590289626 conn(0x7fa774077f50 msgr2=0x7fa7740fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:09.957 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.964+0000 7fa77c491640 1 -- 172.21.15.132:0/1590289626 shutdown_connections 2024-04-23T18:36:09.957 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:09.964+0000 7fa77c491640 1 -- 172.21.15.132:0/1590289626 wait complete. 2024-04-23T18:36:09.967 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:36:10.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:09 smithi148 ceph-mon[32311]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:36:10.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:09 smithi148 ceph-mon[32311]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:36:10.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:09 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2350752694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:10.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:09 smithi157 ceph-mon[32580]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:36:10.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:09 smithi157 ceph-mon[32580]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:36:10.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:09 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2350752694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:10.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.077+0000 7fc0381f9640 1 -- 172.21.15.132:0/3246852949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc030101a30 msgr2=0x7fc030105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.077+0000 7fc0381f9640 1 --2- 172.21.15.132:0/3246852949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc030101a30 0x7fc030105a70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc020009900 tx=0x7fc02002a410 comp rx=0 tx=0).stop 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.077+0000 7fc0381f9640 1 -- 172.21.15.132:0/3246852949 shutdown_connections 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.077+0000 7fc0381f9640 1 --2- 172.21.15.132:0/3246852949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc030106150 0x7fc03010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.077+0000 7fc0381f9640 1 --2- 172.21.15.132:0/3246852949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc030101a30 0x7fc030105a70 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.077+0000 7fc0381f9640 1 --2- 172.21.15.132:0/3246852949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030101080 0x7fc030101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.077+0000 7fc0381f9640 1 -- 172.21.15.132:0/3246852949 >> 172.21.15.132:0/3246852949 conn(0x7fc030078000 msgr2=0x7fc030078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.077+0000 7fc0381f9640 1 -- 172.21.15.132:0/3246852949 shutdown_connections 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.077+0000 7fc0381f9640 1 -- 172.21.15.132:0/3246852949 wait complete. 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.077+0000 7fc0381f9640 1 Processor -- start 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.077+0000 7fc0381f9640 1 -- start start 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc0381f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030101080 0x7fc030199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc0381f9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc030106150 0x7fc030199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc0381f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc03019a040 0x7fc0301a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:10.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc0381f9640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc030111090 con 0x7fc030106150 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc0381f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc030110f10 con 0x7fc03019a040 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc0381f9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc030111210 con 0x7fc030101080 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc035794640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc030106150 0x7fc030199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc035794640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc030106150 0x7fc030199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58544/0 (socket says 172.21.15.132:58544) 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc035794640 1 -- 172.21.15.132:0/724518567 learned_addr learned my addr 172.21.15.132:0/724518567 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc036796640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc03019a040 0x7fc0301a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc035f95640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030101080 0x7fc030199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc035794640 1 -- 172.21.15.132:0/724518567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030101080 msgr2=0x7fc030199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc035794640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030101080 0x7fc030199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc035794640 1 -- 172.21.15.132:0/724518567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc03019a040 msgr2=0x7fc0301a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc035794640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc03019a040 0x7fc0301a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc035794640 1 -- 172.21.15.132:0/724518567 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0301a4dc0 con 0x7fc030106150 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc035794640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc030106150 0x7fc030199ab0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc020006010 tx=0x7fc020004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc026ffd640 1 -- 172.21.15.132:0/724518567 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02003f030 con 0x7fc030106150 2024-04-23T18:36:10.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc026ffd640 1 -- 172.21.15.132:0/724518567 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc020003aa0 con 0x7fc030106150 2024-04-23T18:36:10.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc0381f9640 1 -- 172.21.15.132:0/724518567 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0301a5050 con 0x7fc030106150 2024-04-23T18:36:10.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc026ffd640 1 -- 172.21.15.132:0/724518567 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020035070 con 0x7fc030106150 2024-04-23T18:36:10.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.078+0000 7fc0381f9640 1 -- 172.21.15.132:0/724518567 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0301a5590 con 0x7fc030106150 2024-04-23T18:36:10.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.079+0000 7fc0381f9640 1 -- 172.21.15.132:0/724518567 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc030068d70 con 0x7fc030106150 2024-04-23T18:36:10.075 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.080+0000 7fc026ffd640 1 -- 172.21.15.132:0/724518567 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fc020042030 con 0x7fc030106150 2024-04-23T18:36:10.075 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.081+0000 7fc026ffd640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fc01c0a3ec0 0x7fc01c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:10.075 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.081+0000 7fc035f95640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fc01c0a3ec0 0x7fc01c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:10.075 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.081+0000 7fc026ffd640 1 -- 172.21.15.132:0/724518567 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fc0200f11f0 con 0x7fc030106150 2024-04-23T18:36:10.075 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.082+0000 7fc035f95640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fc01c0a3ec0 0x7fc01c0a6380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc02c005fd0 tx=0x7fc02c005e20 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:10.076 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.085+0000 7fc026ffd640 1 -- 172.21.15.132:0/724518567 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0200bd700 con 0x7fc030106150 2024-04-23T18:36:10.251 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.259+0000 7fc0381f9640 1 -- 172.21.15.132:0/724518567 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc030109380 con 0x7fc030106150 2024-04-23T18:36:10.251 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.259+0000 7fc026ffd640 1 -- 172.21.15.132:0/724518567 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fc0200bd000 con 0x7fc030106150 2024-04-23T18:36:10.251 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-04-23T18:36:10.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.261+0000 7fc0381f9640 1 -- 172.21.15.132:0/724518567 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fc01c0a3ec0 msgr2=0x7fc01c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.261+0000 7fc0381f9640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fc01c0a3ec0 0x7fc01c0a6380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc02c005fd0 tx=0x7fc02c005e20 comp rx=0 tx=0).stop 2024-04-23T18:36:10.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.261+0000 7fc0381f9640 1 -- 172.21.15.132:0/724518567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc030106150 msgr2=0x7fc030199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.261+0000 7fc0381f9640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc030106150 0x7fc030199ab0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc020006010 tx=0x7fc020004060 comp rx=0 tx=0).stop 2024-04-23T18:36:10.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.261+0000 7fc0381f9640 1 -- 172.21.15.132:0/724518567 shutdown_connections 2024-04-23T18:36:10.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.261+0000 7fc0381f9640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fc01c0a3ec0 0x7fc01c0a6380 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.261+0000 7fc0381f9640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc03019a040 0x7fc0301a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.261+0000 7fc0381f9640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc030106150 0x7fc030199ab0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.261+0000 7fc0381f9640 1 --2- 172.21.15.132:0/724518567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc030101080 0x7fc030199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.261+0000 7fc0381f9640 1 -- 172.21.15.132:0/724518567 >> 172.21.15.132:0/724518567 conn(0x7fc030078000 msgr2=0x7fc0300ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:10.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.261+0000 7fc0381f9640 1 -- 172.21.15.132:0/724518567 shutdown_connections 2024-04-23T18:36:10.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.261+0000 7fc0381f9640 1 -- 172.21.15.132:0/724518567 wait complete. 2024-04-23T18:36:10.265 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:36:10.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.377+0000 7faba35cd640 1 -- 172.21.15.132:0/1271085709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab9c106160 msgr2=0x7fab9c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.377+0000 7faba35cd640 1 --2- 172.21.15.132:0/1271085709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab9c106160 0x7fab9c10dd30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fab94009900 tx=0x7fab9402a410 comp rx=0 tx=0).stop 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 -- 172.21.15.132:0/1271085709 shutdown_connections 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 --2- 172.21.15.132:0/1271085709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab9c101a40 0x7fab9c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 --2- 172.21.15.132:0/1271085709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab9c106160 0x7fab9c10dd30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 --2- 172.21.15.132:0/1271085709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab9c101090 0x7fab9c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 -- 172.21.15.132:0/1271085709 >> 172.21.15.132:0/1271085709 conn(0x7fab9c077f50 msgr2=0x7fab9c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 -- 172.21.15.132:0/1271085709 shutdown_connections 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 -- 172.21.15.132:0/1271085709 wait complete. 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 Processor -- start 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 -- start start 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab9c101090 0x7fab9c199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab9c106160 0x7fab9c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab9c19a060 0x7fab9c1a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:10.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1110b0 con 0x7fab9c19a060 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c110f30 con 0x7fab9c106160 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.378+0000 7faba35cd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c111230 con 0x7fab9c101090 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba1b6a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab9c19a060 0x7fab9c1a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba1b6a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab9c19a060 0x7fab9c1a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58556/0 (socket says 172.21.15.132:58556) 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba1b6a640 1 -- 172.21.15.132:0/4050152766 learned_addr learned my addr 172.21.15.132:0/4050152766 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba1369640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab9c101090 0x7fab9c199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba0b68640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab9c106160 0x7fab9c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba1b6a640 1 -- 172.21.15.132:0/4050152766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab9c101090 msgr2=0x7fab9c199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba1b6a640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab9c101090 0x7fab9c199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba1b6a640 1 -- 172.21.15.132:0/4050152766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab9c106160 msgr2=0x7fab9c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba1b6a640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab9c106160 0x7fab9c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba1b6a640 1 -- 172.21.15.132:0/4050152766 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9c1a4de0 con 0x7fab9c19a060 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba1b6a640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab9c19a060 0x7fab9c1a4610 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fab94004770 tx=0x7fab94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:10.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7fab9a7fc640 1 -- 172.21.15.132:0/4050152766 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9403f030 con 0x7fab9c19a060 2024-04-23T18:36:10.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7fab9a7fc640 1 -- 172.21.15.132:0/4050152766 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab94003aa0 con 0x7fab9c19a060 2024-04-23T18:36:10.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7fab9a7fc640 1 -- 172.21.15.132:0/4050152766 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94035070 con 0x7fab9c19a060 2024-04-23T18:36:10.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba35cd640 1 -- 172.21.15.132:0/4050152766 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9c1a5070 con 0x7fab9c19a060 2024-04-23T18:36:10.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.379+0000 7faba35cd640 1 -- 172.21.15.132:0/4050152766 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9c1a5560 con 0x7fab9c19a060 2024-04-23T18:36:10.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.380+0000 7faba35cd640 1 -- 172.21.15.132:0/4050152766 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab9c068d70 con 0x7fab9c19a060 2024-04-23T18:36:10.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.381+0000 7fab9a7fc640 1 -- 172.21.15.132:0/4050152766 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fab94042030 con 0x7fab9c19a060 2024-04-23T18:36:10.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.382+0000 7fab9a7fc640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fab840a3e40 0x7fab840a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:10.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.382+0000 7faba1369640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fab840a3e40 0x7fab840a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:10.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.382+0000 7fab9a7fc640 1 -- 172.21.15.132:0/4050152766 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fab940f1390 con 0x7fab9c19a060 2024-04-23T18:36:10.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.382+0000 7faba1369640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fab840a3e40 0x7fab840a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab8c005fd0 tx=0x7fab8c009680 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:10.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.386+0000 7fab9a7fc640 1 -- 172.21.15.132:0/4050152766 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab940bd8a0 con 0x7fab9c19a060 2024-04-23T18:36:10.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.565+0000 7faba35cd640 1 -- 172.21.15.132:0/4050152766 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fab9c101a40 con 0x7fab9c19a060 2024-04-23T18:36:10.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.565+0000 7fab9a7fc640 1 -- 172.21.15.132:0/4050152766 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fab940bd1a0 con 0x7fab9c19a060 2024-04-23T18:36:10.558 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-04-23T18:36:10.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.567+0000 7faba35cd640 1 -- 172.21.15.132:0/4050152766 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fab840a3e40 msgr2=0x7fab840a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.567+0000 7faba35cd640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fab840a3e40 0x7fab840a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab8c005fd0 tx=0x7fab8c009680 comp rx=0 tx=0).stop 2024-04-23T18:36:10.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.567+0000 7faba35cd640 1 -- 172.21.15.132:0/4050152766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab9c19a060 msgr2=0x7fab9c1a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.568+0000 7faba35cd640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab9c19a060 0x7fab9c1a4610 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fab94004770 tx=0x7fab94004270 comp rx=0 tx=0).stop 2024-04-23T18:36:10.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.568+0000 7faba35cd640 1 -- 172.21.15.132:0/4050152766 shutdown_connections 2024-04-23T18:36:10.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.568+0000 7faba35cd640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fab840a3e40 0x7fab840a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.568+0000 7faba35cd640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab9c19a060 0x7fab9c1a4610 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.568+0000 7faba35cd640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab9c106160 0x7fab9c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.568+0000 7faba35cd640 1 --2- 172.21.15.132:0/4050152766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab9c101090 0x7fab9c199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.568+0000 7faba35cd640 1 -- 172.21.15.132:0/4050152766 >> 172.21.15.132:0/4050152766 conn(0x7fab9c077f50 msgr2=0x7fab9c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:10.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.568+0000 7faba35cd640 1 -- 172.21.15.132:0/4050152766 shutdown_connections 2024-04-23T18:36:10.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.568+0000 7faba35cd640 1 -- 172.21.15.132:0/4050152766 wait complete. 2024-04-23T18:36:10.571 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:36:10.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.677+0000 7f218c9a1640 1 -- 172.21.15.132:0/1089597224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21881031e0 msgr2=0x7f21881035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.677+0000 7f218c9a1640 1 --2- 172.21.15.132:0/1089597224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21881031e0 0x7f21881035c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2178009900 tx=0x7f217802a510 comp rx=0 tx=0).stop 2024-04-23T18:36:10.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 -- 172.21.15.132:0/1089597224 shutdown_connections 2024-04-23T18:36:10.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 --2- 172.21.15.132:0/1089597224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f218810c520 0x7f218810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 --2- 172.21.15.132:0/1089597224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2188103b00 0x7f218810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 --2- 172.21.15.132:0/1089597224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21881031e0 0x7f21881035c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 -- 172.21.15.132:0/1089597224 >> 172.21.15.132:0/1089597224 conn(0x7f2188077f50 msgr2=0x7f2188078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:10.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 -- 172.21.15.132:0/1089597224 shutdown_connections 2024-04-23T18:36:10.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 -- 172.21.15.132:0/1089597224 wait complete. 2024-04-23T18:36:10.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 Processor -- start 2024-04-23T18:36:10.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 -- start start 2024-04-23T18:36:10.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21881031e0 0x7f21881986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:10.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2188103b00 0x7f2188198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f218810c520 0x7f218819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.678+0000 7f218c9a1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f21881121b0 con 0x7f218810c520 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f218c9a1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2188112030 con 0x7f21881031e0 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f218c9a1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2188112330 con 0x7f2188103b00 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f2186d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f218810c520 0x7f218819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f2186d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f218810c520 0x7f218819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58568/0 (socket says 172.21.15.132:58568) 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f2186d9d640 1 -- 172.21.15.132:0/555365779 learned_addr learned my addr 172.21.15.132:0/555365779 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f2185d9b640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2188103b00 0x7f2188198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f218659c640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21881031e0 0x7f21881986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f2186d9d640 1 -- 172.21.15.132:0/555365779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2188103b00 msgr2=0x7f2188198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f2186d9d640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2188103b00 0x7f2188198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f2186d9d640 1 -- 172.21.15.132:0/555365779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21881031e0 msgr2=0x7f21881986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f2186d9d640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21881031e0 0x7f21881986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f2186d9d640 1 -- 172.21.15.132:0/555365779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218819d6c0 con 0x7f218810c520 2024-04-23T18:36:10.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f2185d9b640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2188103b00 0x7f2188198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:10.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f218659c640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21881031e0 0x7f21881986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:10.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f2186d9d640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f218810c520 0x7f218819cf90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f218000eaa0 tx=0x7f218000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:10.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f21777fe640 1 -- 172.21.15.132:0/555365779 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218000cdb0 con 0x7f218810c520 2024-04-23T18:36:10.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f21777fe640 1 -- 172.21.15.132:0/555365779 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2180005020 con 0x7f218810c520 2024-04-23T18:36:10.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f21777fe640 1 -- 172.21.15.132:0/555365779 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21800116d0 con 0x7f218810c520 2024-04-23T18:36:10.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f218c9a1640 1 -- 172.21.15.132:0/555365779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218819d9b0 con 0x7f218810c520 2024-04-23T18:36:10.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.679+0000 7f218c9a1640 1 -- 172.21.15.132:0/555365779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21881a5240 con 0x7f218810c520 2024-04-23T18:36:10.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.680+0000 7f218c9a1640 1 -- 172.21.15.132:0/555365779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2188068d70 con 0x7f218810c520 2024-04-23T18:36:10.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.681+0000 7f21777fe640 1 -- 172.21.15.132:0/555365779 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f21800051d0 con 0x7f218810c520 2024-04-23T18:36:10.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.682+0000 7f21777fe640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f21700a3ec0 0x7f21700a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:10.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.682+0000 7f218659c640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f21700a3ec0 0x7f21700a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:10.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.682+0000 7f21777fe640 1 -- 172.21.15.132:0/555365779 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f21800d0c30 con 0x7f218810c520 2024-04-23T18:36:10.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.682+0000 7f218659c640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f21700a3ec0 0x7f21700a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2178000c00 tx=0x7f2178031400 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:10.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.686+0000 7f21777fe640 1 -- 172.21.15.132:0/555365779 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2180016030 con 0x7f218810c520 2024-04-23T18:36:10.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:10 smithi132 ceph-mon[29178]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 184 KiB/s wr, 79 op/s 2024-04-23T18:36:10.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:10 smithi132 ceph-mon[29178]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:36:10.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:10 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2696227550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:36:10.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:10 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1590289626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:36:10.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:10 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/724518567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:36:10.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.901+0000 7f218c9a1640 1 -- 172.21.15.132:0/555365779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f218802d5c0 con 0x7f218810c520 2024-04-23T18:36:10.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.902+0000 7f21777fe640 1 -- 172.21.15.132:0/555365779 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11946 (secure 0 0 0) 0x7f218009d070 con 0x7f218810c520 2024-04-23T18:36:10.894 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:10.894 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:36:01.455959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24460,24425,24394,24491,24515]},"id":1}]} 2024-04-23T18:36:10.894 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 26 2024-04-23T18:36:10.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.904+0000 7f218c9a1640 1 -- 172.21.15.132:0/555365779 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f21700a3ec0 msgr2=0x7f21700a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.904+0000 7f218c9a1640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f21700a3ec0 0x7f21700a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2178000c00 tx=0x7f2178031400 comp rx=0 tx=0).stop 2024-04-23T18:36:10.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.904+0000 7f218c9a1640 1 -- 172.21.15.132:0/555365779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f218810c520 msgr2=0x7f218819cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:10.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.904+0000 7f218c9a1640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f218810c520 0x7f218819cf90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f218000eaa0 tx=0x7f218000ef60 comp rx=0 tx=0).stop 2024-04-23T18:36:10.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.904+0000 7f218c9a1640 1 -- 172.21.15.132:0/555365779 shutdown_connections 2024-04-23T18:36:10.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.904+0000 7f218c9a1640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f21700a3ec0 0x7f21700a6380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.904+0000 7f218c9a1640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2188103b00 0x7f2188198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.904+0000 7f218c9a1640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f218810c520 0x7f218819cf90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.904+0000 7f218c9a1640 1 --2- 172.21.15.132:0/555365779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21881031e0 0x7f21881986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:10.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.904+0000 7f218c9a1640 1 -- 172.21.15.132:0/555365779 >> 172.21.15.132:0/555365779 conn(0x7f2188077f50 msgr2=0x7f21880fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:10.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.904+0000 7f218c9a1640 1 -- 172.21.15.132:0/555365779 shutdown_connections 2024-04-23T18:36:10.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:10.904+0000 7f218c9a1640 1 -- 172.21.15.132:0/555365779 wait complete. 2024-04-23T18:36:10.908 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:36:11.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.014+0000 7f330af0f640 1 -- 172.21.15.132:0/991911062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3304106170 msgr2=0x7f330410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.014+0000 7f330af0f640 1 --2- 172.21.15.132:0/991911062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3304106170 0x7f330410dd40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009ca0 tx=0x7f32fc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.014+0000 7f330af0f640 1 -- 172.21.15.132:0/991911062 shutdown_connections 2024-04-23T18:36:11.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.014+0000 7f330af0f640 1 --2- 172.21.15.132:0/991911062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3304101a50 0x7f3304105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.014+0000 7f330af0f640 1 --2- 172.21.15.132:0/991911062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3304106170 0x7f330410dd40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.014+0000 7f330af0f640 1 --2- 172.21.15.132:0/991911062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33041010a0 0x7f3304101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.014+0000 7f330af0f640 1 -- 172.21.15.132:0/991911062 >> 172.21.15.132:0/991911062 conn(0x7f3304077f50 msgr2=0x7f3304078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.014+0000 7f330af0f640 1 -- 172.21.15.132:0/991911062 shutdown_connections 2024-04-23T18:36:11.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.014+0000 7f330af0f640 1 -- 172.21.15.132:0/991911062 wait complete. 2024-04-23T18:36:11.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f330af0f640 1 Processor -- start 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f330af0f640 1 -- start start 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f330af0f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33041010a0 0x7f3304102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f330af0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3304106170 0x7f3304103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f330af0f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33041035b0 0x7f33041a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f330af0f640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f33041110e0 con 0x7f33041035b0 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f330af0f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3304110f60 con 0x7f3304106170 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f330af0f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3304111260 con 0x7f33041010a0 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f33094ac640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33041035b0 0x7f33041a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f33094ac640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33041035b0 0x7f33041a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58600/0 (socket says 172.21.15.132:58600) 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f33094ac640 1 -- 172.21.15.132:0/2796217688 learned_addr learned my addr 172.21.15.132:0/2796217688 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f3308cab640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33041010a0 0x7f3304102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f3303fff640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3304106170 0x7f3304103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f33094ac640 1 -- 172.21.15.132:0/2796217688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33041010a0 msgr2=0x7f3304102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f33094ac640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33041010a0 0x7f3304102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f33094ac640 1 -- 172.21.15.132:0/2796217688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3304106170 msgr2=0x7f3304103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f33094ac640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3304106170 0x7f3304103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f33094ac640 1 -- 172.21.15.132:0/2796217688 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33041a7020 con 0x7f33041035b0 2024-04-23T18:36:11.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.015+0000 7f3303fff640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3304106170 0x7f3304103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:11.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.016+0000 7f33094ac640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33041035b0 0x7f33041a6920 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f32fc005cc0 tx=0x7f32fc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.016+0000 7f3301ffb640 1 -- 172.21.15.132:0/2796217688 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc003bf0 con 0x7f33041035b0 2024-04-23T18:36:11.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.016+0000 7f3301ffb640 1 -- 172.21.15.132:0/2796217688 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32fc003d90 con 0x7f33041035b0 2024-04-23T18:36:11.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.016+0000 7f3301ffb640 1 -- 172.21.15.132:0/2796217688 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc02fbc0 con 0x7f33041035b0 2024-04-23T18:36:11.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.016+0000 7f330af0f640 1 -- 172.21.15.132:0/2796217688 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33041a72b0 con 0x7f33041035b0 2024-04-23T18:36:11.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.016+0000 7f330af0f640 1 -- 172.21.15.132:0/2796217688 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33041a7800 con 0x7f33041035b0 2024-04-23T18:36:11.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.017+0000 7f330af0f640 1 -- 172.21.15.132:0/2796217688 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3304068d70 con 0x7f33041035b0 2024-04-23T18:36:11.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.018+0000 7f3301ffb640 1 -- 172.21.15.132:0/2796217688 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f32fc02fd60 con 0x7f33041035b0 2024-04-23T18:36:11.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.018+0000 7f3301ffb640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f32f40a3e70 0x7f32f40a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.019+0000 7f3308cab640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f32f40a3e70 0x7f32f40a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.019+0000 7f3301ffb640 1 -- 172.21.15.132:0/2796217688 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f32fc0f21a0 con 0x7f33041035b0 2024-04-23T18:36:11.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.019+0000 7f3308cab640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f32f40a3e70 0x7f32f40a6330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32f8005fd0 tx=0x7f32f8005850 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:10 smithi148 ceph-mon[32311]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 184 KiB/s wr, 79 op/s 2024-04-23T18:36:11.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:10 smithi148 ceph-mon[32311]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:36:11.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:10 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2696227550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:36:11.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:10 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1590289626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:36:11.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:10 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/724518567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:36:11.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.022+0000 7f3301ffb640 1 -- 172.21.15.132:0/2796217688 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32fc0be690 con 0x7f33041035b0 2024-04-23T18:36:11.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:10 smithi157 ceph-mon[32580]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 184 KiB/s wr, 79 op/s 2024-04-23T18:36:11.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:10 smithi157 ceph-mon[32580]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:36:11.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:10 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2696227550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:36:11.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:10 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1590289626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:36:11.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:10 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/724518567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:36:11.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.201+0000 7f330af0f640 1 -- 172.21.15.132:0/2796217688 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3304101af0 con 0x7f33041035b0 2024-04-23T18:36:11.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.202+0000 7f3301ffb640 1 -- 172.21.15.132:0/2796217688 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23228 (secure 0 0 0) 0x7f32fc0098b0 con 0x7f33041035b0 2024-04-23T18:36:11.195 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:11.195 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":91,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:36:02.717304+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/2124409853":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/3095429281":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/899407689":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1857586348":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:36:11.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.205+0000 7f330af0f640 1 -- 172.21.15.132:0/2796217688 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f32f40a3e70 msgr2=0x7f32f40a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.205+0000 7f330af0f640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f32f40a3e70 0x7f32f40a6330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32f8005fd0 tx=0x7f32f8005850 comp rx=0 tx=0).stop 2024-04-23T18:36:11.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.205+0000 7f330af0f640 1 -- 172.21.15.132:0/2796217688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33041035b0 msgr2=0x7f33041a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.205+0000 7f330af0f640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33041035b0 0x7f33041a6920 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f32fc005cc0 tx=0x7f32fc02f380 comp rx=0 tx=0).stop 2024-04-23T18:36:11.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.205+0000 7f330af0f640 1 -- 172.21.15.132:0/2796217688 shutdown_connections 2024-04-23T18:36:11.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.205+0000 7f330af0f640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f32f40a3e70 0x7f32f40a6330 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.205+0000 7f330af0f640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33041035b0 0x7f33041a6920 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.205+0000 7f330af0f640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3304106170 0x7f3304103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.205+0000 7f330af0f640 1 --2- 172.21.15.132:0/2796217688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33041010a0 0x7f3304102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.205+0000 7f330af0f640 1 -- 172.21.15.132:0/2796217688 >> 172.21.15.132:0/2796217688 conn(0x7f3304077f50 msgr2=0x7f3304104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.205+0000 7f330af0f640 1 -- 172.21.15.132:0/2796217688 shutdown_connections 2024-04-23T18:36:11.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.205+0000 7f330af0f640 1 -- 172.21.15.132:0/2796217688 wait complete. 2024-04-23T18:36:11.209 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 476737 2024-04-23T18:36:11.209 DEBUG:tasks.quiescer:Yielding 2024-04-23T18:36:11.209 INFO:teuthology.run_tasks:Running task workunit... 2024-04-23T18:36:11.227 INFO:tasks.workunit:Pulling workunits from ref 57e0caf76fdb7ab8b1358588f292d08519163844 2024-04-23T18:36:11.228 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-23T18:36:11.228 INFO:tasks.workunit:timeout=3h 2024-04-23T18:36:11.228 INFO:tasks.workunit:cleanup=True 2024-04-23T18:36:11.228 DEBUG:teuthology.orchestra.run.smithi132:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-23T18:36:11.229 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:36:11.243 INFO:teuthology.orchestra.run.smithi132.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-23T18:36:11.243 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-23T18:36:11.243 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-23T18:36:11.243 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:36:11.244 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-23T18:36:11.244 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-23 18:34:07.587057894 +0000 2024-04-23T18:36:11.244 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-23 18:34:07.587057894 +0000 2024-04-23T18:36:11.244 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-23 18:36:00.803393337 +0000 2024-04-23T18:36:11.244 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-23 18:34:07.587057894 +0000 2024-04-23T18:36:11.244 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-23T18:36:11.244 DEBUG:teuthology.orchestra.run.smithi132:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-23T18:36:11.316 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 57e0caf76fdb7ab8b1358588f292d08519163844 2024-04-23T18:36:11.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.337+0000 7ff74267c640 1 -- 172.21.15.132:0/4162589204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff73c106190 msgr2=0x7ff73c10dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.337+0000 7ff74267c640 1 --2- 172.21.15.132:0/4162589204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff73c106190 0x7ff73c10dd60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff734009ca0 tx=0x7ff73402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.337+0000 7ff74267c640 1 -- 172.21.15.132:0/4162589204 shutdown_connections 2024-04-23T18:36:11.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.337+0000 7ff74267c640 1 --2- 172.21.15.132:0/4162589204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff73c106190 0x7ff73c10dd60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.337+0000 7ff74267c640 1 --2- 172.21.15.132:0/4162589204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff73c101ac0 0x7ff73c105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.337+0000 7ff74267c640 1 --2- 172.21.15.132:0/4162589204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff73c101110 0x7ff73c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.337+0000 7ff74267c640 1 -- 172.21.15.132:0/4162589204 >> 172.21.15.132:0/4162589204 conn(0x7ff73c077fb0 msgr2=0x7ff73c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.337+0000 7ff74267c640 1 -- 172.21.15.132:0/4162589204 shutdown_connections 2024-04-23T18:36:11.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.337+0000 7ff74267c640 1 -- 172.21.15.132:0/4162589204 wait complete. 2024-04-23T18:36:11.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff74267c640 1 Processor -- start 2024-04-23T18:36:11.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff74267c640 1 -- start start 2024-04-23T18:36:11.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff74267c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff73c101110 0x7ff73c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff74267c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff73c106190 0x7ff73c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff74267c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff73c19cf80 0x7ff73c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff74267c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c112210 con 0x7ff73c19cf80 2024-04-23T18:36:11.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff74267c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c112090 con 0x7ff73c106190 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff74267c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c112390 con 0x7ff73c101110 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff740c19640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff73c19cf80 0x7ff73c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff740c19640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff73c19cf80 0x7ff73c19d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58614/0 (socket says 172.21.15.132:58614) 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff740c19640 1 -- 172.21.15.132:0/1308127461 learned_addr learned my addr 172.21.15.132:0/1308127461 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff73b7fe640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff73c106190 0x7ff73c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.338+0000 7ff73bfff640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff73c101110 0x7ff73c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.339+0000 7ff740c19640 1 -- 172.21.15.132:0/1308127461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff73c101110 msgr2=0x7ff73c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.339+0000 7ff740c19640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff73c101110 0x7ff73c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.339+0000 7ff740c19640 1 -- 172.21.15.132:0/1308127461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff73c106190 msgr2=0x7ff73c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.339+0000 7ff740c19640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff73c106190 0x7ff73c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.339+0000 7ff740c19640 1 -- 172.21.15.132:0/1308127461 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73c19da70 con 0x7ff73c19cf80 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.339+0000 7ff740c19640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff73c19cf80 0x7ff73c19d430 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff734005cc0 tx=0x7ff7340316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.339+0000 7ff7397fa640 1 -- 172.21.15.132:0/1308127461 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734003b70 con 0x7ff73c19cf80 2024-04-23T18:36:11.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.339+0000 7ff74267c640 1 -- 172.21.15.132:0/1308127461 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73c19dd00 con 0x7ff73c19cf80 2024-04-23T18:36:11.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.339+0000 7ff7397fa640 1 -- 172.21.15.132:0/1308127461 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7340318d0 con 0x7ff73c19cf80 2024-04-23T18:36:11.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.339+0000 7ff7397fa640 1 -- 172.21.15.132:0/1308127461 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73402f8c0 con 0x7ff73c19cf80 2024-04-23T18:36:11.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.339+0000 7ff74267c640 1 -- 172.21.15.132:0/1308127461 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff73c1a5500 con 0x7ff73c19cf80 2024-04-23T18:36:11.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.340+0000 7ff74267c640 1 -- 172.21.15.132:0/1308127461 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff73c0684e0 con 0x7ff73c19cf80 2024-04-23T18:36:11.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.341+0000 7ff7397fa640 1 -- 172.21.15.132:0/1308127461 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7ff73402faa0 con 0x7ff73c19cf80 2024-04-23T18:36:11.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.342+0000 7ff7397fa640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff7240a3e40 0x7ff7240a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.342+0000 7ff7397fa640 1 -- 172.21.15.132:0/1308127461 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7ff7340f1da0 con 0x7ff73c19cf80 2024-04-23T18:36:11.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.342+0000 7ff73bfff640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff7240a3e40 0x7ff7240a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.342+0000 7ff73bfff640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff7240a3e40 0x7ff7240a6300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009bd0 tx=0x7ff72c005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.347+0000 7ff7397fa640 1 -- 172.21.15.132:0/1308127461 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7340be2f0 con 0x7ff73c19cf80 2024-04-23T18:36:11.344 INFO:tasks.workunit.client.0.smithi132.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-23T18:36:11.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.568+0000 7ff74267c640 1 -- 172.21.15.132:0/1308127461 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff73c109480 con 0x7ff73c19cf80 2024-04-23T18:36:11.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.569+0000 7ff7397fa640 1 -- 172.21.15.132:0/1308127461 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11946 (secure 0 0 0) 0x7ff7340bdbf0 con 0x7ff73c19cf80 2024-04-23T18:36:11.561 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:11.561 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:36:01.455959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24460,24425,24394,24491,24515]},"id":1}]} 2024-04-23T18:36:11.561 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 26 2024-04-23T18:36:11.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.571+0000 7ff74267c640 1 -- 172.21.15.132:0/1308127461 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff7240a3e40 msgr2=0x7ff7240a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.571+0000 7ff74267c640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff7240a3e40 0x7ff7240a6300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009bd0 tx=0x7ff72c005f90 comp rx=0 tx=0).stop 2024-04-23T18:36:11.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.571+0000 7ff74267c640 1 -- 172.21.15.132:0/1308127461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff73c19cf80 msgr2=0x7ff73c19d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.571+0000 7ff74267c640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff73c19cf80 0x7ff73c19d430 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff734005cc0 tx=0x7ff7340316e0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.571+0000 7ff74267c640 1 -- 172.21.15.132:0/1308127461 shutdown_connections 2024-04-23T18:36:11.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.571+0000 7ff74267c640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff7240a3e40 0x7ff7240a6300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.571+0000 7ff74267c640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff73c19cf80 0x7ff73c19d430 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.571+0000 7ff74267c640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff73c106190 0x7ff73c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.571+0000 7ff74267c640 1 --2- 172.21.15.132:0/1308127461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff73c101110 0x7ff73c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.571+0000 7ff74267c640 1 -- 172.21.15.132:0/1308127461 >> 172.21.15.132:0/1308127461 conn(0x7ff73c077fb0 msgr2=0x7ff73c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.571+0000 7ff74267c640 1 -- 172.21.15.132:0/1308127461 shutdown_connections 2024-04-23T18:36:11.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.571+0000 7ff74267c640 1 -- 172.21.15.132:0/1308127461 wait complete. 2024-04-23T18:36:11.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T18:34:00.571339+0000', 'modified': '2024-04-23T18:36:01.455959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24515, 'mds_1': 24491, 'mds_2': 24460, 'mds_3': 24425, 'mds_4': 24394}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6839/579251026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 579251026}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 579251026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6841/826931131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6840', 'nonce': 826931131}, {'type': 'v1', 'addr': '172.21.15.132:6841', 'nonce': 826931131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24394': {'gid': 24394, 'name': 'l', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.157:6833/1034092547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 1034092547}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 1034092547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.148:6833/3556073608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 3556073608}, {'type': 'v1', 'addr': '172.21.15.148:6833', 'nonce': 3556073608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/4121037781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 4121037781}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 4121037781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6837/2670322328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 2670322328}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 2670322328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/397710093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 397710093}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 397710093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.157:6839/1528717147', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 1528717147}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 1528717147}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.148:6837/2626530887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 2626530887}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 2626530887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6839/4115412726', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 4115412726}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 4115412726}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24515, 'qdb_cluster': [24460, 24425, 24394, 24491, 24515]} 2024-04-23T18:36:11.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-23T18:36:11.576 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-23T18:36:11.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.723+0000 7f8a39bf7640 1 -- 172.21.15.132:0/3185223777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a34107250 msgr2=0x7f8a3410ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.723+0000 7f8a39bf7640 1 --2- 172.21.15.132:0/3185223777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a34107250 0x7f8a3410ee20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009ca0 tx=0x7f8a2c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.723+0000 7f8a39bf7640 1 -- 172.21.15.132:0/3185223777 shutdown_connections 2024-04-23T18:36:11.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.723+0000 7f8a39bf7640 1 --2- 172.21.15.132:0/3185223777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a34102b30 0x7f8a34106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.723+0000 7f8a39bf7640 1 --2- 172.21.15.132:0/3185223777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a34107250 0x7f8a3410ee20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.723+0000 7f8a39bf7640 1 --2- 172.21.15.132:0/3185223777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a34102180 0x7f8a34102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.723+0000 7f8a39bf7640 1 -- 172.21.15.132:0/3185223777 >> 172.21.15.132:0/3185223777 conn(0x7f8a340792d0 msgr2=0x7f8a340796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.723+0000 7f8a39bf7640 1 -- 172.21.15.132:0/3185223777 shutdown_connections 2024-04-23T18:36:11.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.723+0000 7f8a39bf7640 1 -- 172.21.15.132:0/3185223777 wait complete. 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.724+0000 7f8a39bf7640 1 Processor -- start 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.724+0000 7f8a39bf7640 1 -- start start 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.724+0000 7f8a39bf7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a34102180 0x7f8a34198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.724+0000 7f8a39bf7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a34107250 0x7f8a34198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.724+0000 7f8a39bf7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a3419cfb0 0x7f8a3419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.724+0000 7f8a39bf7640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341121c0 con 0x7f8a34107250 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.724+0000 7f8a39bf7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34112040 con 0x7f8a3419cfb0 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.724+0000 7f8a39bf7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34112340 con 0x7f8a34102180 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.724+0000 7f8a32ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a34107250 0x7f8a34198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.724+0000 7f8a32ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a34107250 0x7f8a34198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58634/0 (socket says 172.21.15.132:58634) 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.724+0000 7f8a32ffd640 1 -- 172.21.15.132:0/2183023859 learned_addr learned my addr 172.21.15.132:0/2183023859 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a337fe640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a34102180 0x7f8a34198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a33fff640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a3419cfb0 0x7f8a3419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a32ffd640 1 -- 172.21.15.132:0/2183023859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a34102180 msgr2=0x7f8a34198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a32ffd640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a34102180 0x7f8a34198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a32ffd640 1 -- 172.21.15.132:0/2183023859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a3419cfb0 msgr2=0x7f8a3419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a32ffd640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a3419cfb0 0x7f8a3419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a32ffd640 1 -- 172.21.15.132:0/2183023859 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3419daa0 con 0x7f8a34107250 2024-04-23T18:36:11.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a32ffd640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a34107250 0x7f8a34198c80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800c970 tx=0x7f8a2800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a30ff9640 1 -- 172.21.15.132:0/2183023859 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28007bb0 con 0x7f8a34107250 2024-04-23T18:36:11.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a39bf7640 1 -- 172.21.15.132:0/2183023859 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3419dd90 con 0x7f8a34107250 2024-04-23T18:36:11.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a30ff9640 1 -- 172.21.15.132:0/2183023859 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a28007d50 con 0x7f8a34107250 2024-04-23T18:36:11.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a30ff9640 1 -- 172.21.15.132:0/2183023859 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28018730 con 0x7f8a34107250 2024-04-23T18:36:11.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.725+0000 7f8a39bf7640 1 -- 172.21.15.132:0/2183023859 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a341a5530 con 0x7f8a34107250 2024-04-23T18:36:11.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.727+0000 7f8a30ff9640 1 -- 172.21.15.132:0/2183023859 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f8a2800f990 con 0x7f8a34107250 2024-04-23T18:36:11.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a3c9e9740 1 Processor -- start 2024-04-23T18:36:11.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a3c9e9740 1 -- start start 2024-04-23T18:36:11.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a3c9e9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 0x5582bfdce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a3c9e9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdce760 0x5582bfdd8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a30ff9640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8a1c0a3e40 0x7f8a1c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a3c9e9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdd8dd0 0x5582bfddb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a3c9e9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5582bfd33c50 con 0x5582bfdcddb0 2024-04-23T18:36:11.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a3c9e9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5582bfcd0830 con 0x5582bfdce760 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a3c9e9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5582bfd2fc10 con 0x5582bfdd8dd0 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 0x5582bfdce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a32ffd640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8a1c0a3e40 0x7f8a1c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 0x5582bfdce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58640/0 (socket says 172.21.15.132:58640) 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a33fff640 1 -- 172.21.15.132:0/3220637986 learned_addr learned my addr 172.21.15.132:0/3220637986 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a30ff9640 1 -- 172.21.15.132:0/2183023859 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f8a280d0c20 con 0x7f8a34107250 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a33fff640 1 --2- 172.21.15.132:0/3220637986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdd8dd0 0x5582bfddb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a337fe640 1 --2- 172.21.15.132:0/3220637986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdce760 0x5582bfdd8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a33fff640 1 -- 172.21.15.132:0/3220637986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdd8dd0 msgr2=0x5582bfddb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a33fff640 1 --2- 172.21.15.132:0/3220637986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdd8dd0 0x5582bfddb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a33fff640 1 -- 172.21.15.132:0/3220637986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdce760 msgr2=0x5582bfdd8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a33fff640 1 --2- 172.21.15.132:0/3220637986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdce760 0x5582bfdd8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a33fff640 1 -- 172.21.15.132:0/3220637986 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bfa462d0 con 0x5582bfdcddb0 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a32ffd640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8a1c0a3e40 0x7f8a1c0a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a280ce1e0 tx=0x7f8a2806f4a0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.728+0000 7f8a33fff640 1 --2- 172.21.15.132:0/3220637986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 0x5582bfdce190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009dc0 tx=0x7f8a2c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e3c7298ec38d9d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.729+0000 7f8a19ffb640 1 -- 172.21.15.132:0/3220637986 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c031650 con 0x5582bfdcddb0 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.729+0000 7f8a19ffb640 1 -- 172.21.15.132:0/3220637986 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2c045070 con 0x5582bfdcddb0 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.729+0000 7f8a19ffb640 1 -- 172.21.15.132:0/3220637986 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0045d0 con 0x5582bfdcddb0 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.729+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/3220637986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 msgr2=0x5582bfdce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.729+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/3220637986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 0x5582bfdce190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009dc0 tx=0x7f8a2c0093d0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.729+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/3220637986 shutdown_connections 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.729+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/3220637986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdd8dd0 0x5582bfddb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.729+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/3220637986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdce760 0x5582bfdd8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.729+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/3220637986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 0x5582bfdce190 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.729+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/3220637986 >> 172.21.15.132:0/3220637986 conn(0x5582bfdca070 msgr2=0x5582bfdca450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.729+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/3220637986 shutdown_connections 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.729+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/3220637986 wait complete. 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a3c9e9740 1 -- start start 2024-04-23T18:36:11.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a3c9e9740 1 Processor -- start 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a3c9e9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdcddb0 0x5582bfdd5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a3c9e9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a3c9e9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a3c9e9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5582bfddbce0 con 0x5582bfdd8dd0 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a3c9e9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5582bfcce470 con 0x5582bfdcddb0 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a3c9e9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5582bfddbe60 con 0x5582bfdce760 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a3c9e9740 10 client.0 _ll_register_callbacks cb 0x5582bfcf3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58646/0 (socket says 172.21.15.132:58646) 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a33fff640 1 -- 172.21.15.132:0/1232402799 learned_addr learned my addr 172.21.15.132:0/1232402799 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a33fff640 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdcddb0 0x5582bfdd5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.730+0000 7f8a337fe640 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a33fff640 1 -- 172.21.15.132:0/1232402799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 msgr2=0x5582bfdd61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a33fff640 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a33fff640 1 -- 172.21.15.132:0/1232402799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdcddb0 msgr2=0x5582bfdd5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a33fff640 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdcddb0 0x5582bfdd5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a33fff640 1 -- 172.21.15.132:0/1232402799 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bfb106b0 con 0x5582bfdd8dd0 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a337fe640 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a33fff640 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd6710 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c02f350 tx=0x7f8a2c02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0892049b348eff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a12ffd640 1 -- 172.21.15.132:0/1232402799 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c003b70 con 0x5582bfdd8dd0 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a12ffd640 1 -- 172.21.15.132:0/1232402799 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f8a2c003d10 con 0x5582bfdd8dd0 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/1232402799 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a2c045070 con 0x5582bfdd8dd0 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a12ffd640 1 -- 172.21.15.132:0/1232402799 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a2c003f10 con 0x5582bfdd8dd0 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a3c9e9740 10 client.15594 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/1232402799 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5582bfddbce0 con 0x5582bfdd8dd0 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a12ffd640 1 -- 172.21.15.132:0/1232402799 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2c045070 con 0x5582bfdd8dd0 2024-04-23T18:36:11.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a12ffd640 1 -- 172.21.15.132:0/1232402799 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8a2c0305c0 con 0x5582bfdd8dd0 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a3c9e9740 10 client.15594 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a3c9e9740 10 client.15594 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a3c9e9740 20 client.15594 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x5582bfdea0e0 0x5582bfdeb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a3c9e9740 4 client.15594 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.731+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/1232402799 --> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582bfd41670 con 0x5582bfdea0e0 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.732+0000 7f8a337fe640 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x5582bfdea0e0 0x5582bfdeb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.732+0000 7f8a12ffd640 10 client.15594 ms_handle_connect on v2:172.21.15.157:6832/1034092547 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.732+0000 7f8a337fe640 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x5582bfdea0e0 0x5582bfdeb550 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9f151e534aa1ed5c server_cookie=35c1eaee4546aee2 in_seq=0 out_seq=0 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a12ffd640 1 -- 172.21.15.132:0/1232402799 <== mds.4 v2:172.21.15.157:6832/1034092547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582bfd41670 con 0x5582bfdea0e0 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a12ffd640 10 client.15594 handle_command_reply: tid=0 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a3c9e9740 1 client.15594 shutdown 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/1232402799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 msgr2=0x5582bfdd6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd6710 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c02f350 tx=0x7f8a2c02bfd0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/1232402799 shutdown_connections 2024-04-23T18:36:11.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x5582bfdea0e0 0x5582bfdeb550 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd6710 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a12ffd640 0 client.15594 ms_handle_reset on v2:172.21.15.157:6832/1034092547 2024-04-23T18:36:11.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/1232402799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdcddb0 0x5582bfdd5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/1232402799 >> 172.21.15.132:0/1232402799 conn(0x5582bfdca070 msgr2=0x5582bfdcfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/1232402799 shutdown_connections 2024-04-23T18:36:11.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/1232402799 wait complete. 2024-04-23T18:36:11.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.733+0000 7f8a3c9e9740 20 client.15594 trim_cache size 0 max 16384 2024-04-23T18:36:11.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a3c9e9740 1 Processor -- start 2024-04-23T18:36:11.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a3c9e9740 1 -- start start 2024-04-23T18:36:11.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a3c9e9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdcddb0 0x5582bfdc9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a3c9e9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a3c9e9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a3c9e9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5582bfde9890 con 0x5582bfdd8dd0 2024-04-23T18:36:11.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a3c9e9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5582bfde9f60 con 0x5582bfdcddb0 2024-04-23T18:36:11.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a3c9e9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5582bfdea630 con 0x5582bfdce760 2024-04-23T18:36:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58656/0 (socket says 172.21.15.132:58656) 2024-04-23T18:36:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a33fff640 1 -- 172.21.15.132:0/3079451314 learned_addr learned my addr 172.21.15.132:0/3079451314 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a33fff640 1 --2- 172.21.15.132:0/3079451314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdcddb0 0x5582bfdc9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a337fe640 1 --2- 172.21.15.132:0/3079451314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a33fff640 1 -- 172.21.15.132:0/3079451314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 msgr2=0x5582bfdd05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a33fff640 1 --2- 172.21.15.132:0/3079451314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a33fff640 1 -- 172.21.15.132:0/3079451314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdcddb0 msgr2=0x5582bfdc9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a33fff640 1 --2- 172.21.15.132:0/3079451314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdcddb0 0x5582bfdc9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a33fff640 1 -- 172.21.15.132:0/3079451314 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bfdc59d0 con 0x5582bfdd8dd0 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.752+0000 7f8a33fff640 1 --2- 172.21.15.132:0/3079451314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd2f10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x5582bfd79620 tx=0x7f8a2c040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650a2652e68cfb3b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.753+0000 7f8a197fa640 1 -- 172.21.15.132:0/3079451314 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c05e070 con 0x5582bfdd8dd0 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.753+0000 7f8a197fa640 1 -- 172.21.15.132:0/3079451314 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2c045070 con 0x5582bfdd8dd0 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.753+0000 7f8a197fa640 1 -- 172.21.15.132:0/3079451314 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c030bb0 con 0x5582bfdd8dd0 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.753+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/3079451314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 msgr2=0x5582bfdd2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.753+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/3079451314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd2f10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x5582bfd79620 tx=0x7f8a2c040040 comp rx=0 tx=0).stop 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.753+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/3079451314 shutdown_connections 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.753+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/3079451314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.753+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/3079451314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd2f10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.753+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/3079451314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdcddb0 0x5582bfdc9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.753+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/3079451314 >> 172.21.15.132:0/3079451314 conn(0x5582bfdca070 msgr2=0x5582bfdd4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.753+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/3079451314 shutdown_connections 2024-04-23T18:36:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.753+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/3079451314 wait complete. 2024-04-23T18:36:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a3c9e9740 1 -- start start 2024-04-23T18:36:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a3c9e9740 1 Processor -- start 2024-04-23T18:36:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a3c9e9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 0x5582bfdd2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a3c9e9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a3c9e9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a3c9e9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5582bfdeb3d0 con 0x5582bfdcddb0 2024-04-23T18:36:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a3c9e9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5582bfdead00 con 0x5582bfdd8dd0 2024-04-23T18:36:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a3c9e9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5582bfdebaa0 con 0x5582bfdce760 2024-04-23T18:36:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a3c9e9740 10 client.0 _ll_register_callbacks cb 0x7f8a2c031b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 0x5582bfdd2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 0x5582bfdd2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58664/0 (socket says 172.21.15.132:58664) 2024-04-23T18:36:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a33fff640 1 -- 172.21.15.132:0/68875104 learned_addr learned my addr 172.21.15.132:0/68875104 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a337fe640 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a33fff640 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a33fff640 1 -- 172.21.15.132:0/68875104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 msgr2=0x5582bfdd31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a33fff640 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a33fff640 1 -- 172.21.15.132:0/68875104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdd8dd0 msgr2=0x5582bfdd3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a33fff640 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.754+0000 7f8a33fff640 1 -- 172.21.15.132:0/68875104 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bfdf1980 con 0x5582bfdcddb0 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a33fff640 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 0x5582bfdd2ca0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c030940 tx=0x7f8a2c031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcaff5f0d5badd86 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a12ffd640 1 -- 172.21.15.132:0/68875104 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c05e070 con 0x5582bfdcddb0 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a12ffd640 1 -- 172.21.15.132:0/68875104 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f8a2c003970 con 0x5582bfdcddb0 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/68875104 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a2c045070 con 0x5582bfdcddb0 2024-04-23T18:36:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a12ffd640 1 -- 172.21.15.132:0/68875104 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a2c003b90 con 0x5582bfdcddb0 2024-04-23T18:36:11.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a3c9e9740 10 client.15600 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:11.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/68875104 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5582bfe50340 con 0x5582bfdcddb0 2024-04-23T18:36:11.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a12ffd640 1 -- 172.21.15.132:0/68875104 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2c045070 con 0x5582bfdcddb0 2024-04-23T18:36:11.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a12ffd640 1 -- 172.21.15.132:0/68875104 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8a2c062020 con 0x5582bfdcddb0 2024-04-23T18:36:11.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a3c9e9740 10 client.15600 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:11.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a3c9e9740 10 client.15600 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-23T18:36:11.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a3c9e9740 20 client.15600 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:11.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x5582bfe54a50 0x5582bfe53760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a3c9e9740 4 client.15600 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.755+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/68875104 --> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] -- command(tid 0: {"prefix": "status"}) -- 0x5582bfd3b4e0 con 0x5582bfe54a50 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a337fe640 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x5582bfe54a50 0x5582bfe53760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a12ffd640 10 client.15600 ms_handle_connect on v2:172.21.15.157:6832/1034092547 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a337fe640 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x5582bfe54a50 0x5582bfe53760 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a134ed7c2ed5cc64 server_cookie=c50c2306ab71cdfd in_seq=0 out_seq=0 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a12ffd640 1 -- 172.21.15.132:0/68875104 <== mds.4 v2:172.21.15.157:6832/1034092547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5582bfd3b4e0 con 0x5582bfe54a50 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a12ffd640 10 client.15600 handle_command_reply: tid=0 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a3c9e9740 1 client.15600 shutdown 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/68875104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 msgr2=0x5582bfdd2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 0x5582bfdd2ca0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c030940 tx=0x7f8a2c031760 comp rx=0 tx=0).stop 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/68875104 shutdown_connections 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.157:6832/1034092547,v1:172.21.15.157:6833/1034092547] conn(0x5582bfe54a50 0x5582bfe53760 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5582bfdce760 0x5582bfdd31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582bfdd8dd0 0x5582bfdd3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a12ffd640 0 client.15600 ms_handle_reset on v2:172.21.15.157:6832/1034092547 2024-04-23T18:36:11.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a3c9e9740 1 --2- 172.21.15.132:0/68875104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582bfdcddb0 0x5582bfdd2ca0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/68875104 >> 172.21.15.132:0/68875104 conn(0x5582bfdca070 msgr2=0x5582bfde40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/68875104 shutdown_connections 2024-04-23T18:36:11.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.756+0000 7f8a3c9e9740 1 -- 172.21.15.132:0/68875104 wait complete. 2024-04-23T18:36:11.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a3c9e9740 20 client.15600 trim_cache size 0 max 16384 2024-04-23T18:36:11.763 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:36:11.763 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:36:11.763 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 4, 2024-04-23T18:36:11.764 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24394, 2024-04-23T18:36:11.764 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:36:11.764 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:36:11.764 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:36:11.764 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 125.222994966, 2024-04-23T18:36:11.764 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 26, 2024-04-23T18:36:11.764 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 91, 2024-04-23T18:36:11.764 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 90, 2024-04-23T18:36:11.764 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 169.01776083999999 2024-04-23T18:36:11.764 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:36:11.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a39bf7640 1 -- 172.21.15.132:0/2183023859 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8a1c0a3e40 msgr2=0x7f8a1c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a39bf7640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8a1c0a3e40 0x7f8a1c0a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a280ce1e0 tx=0x7f8a2806f4a0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a39bf7640 1 -- 172.21.15.132:0/2183023859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a34107250 msgr2=0x7f8a34198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a39bf7640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a34107250 0x7f8a34198c80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800c970 tx=0x7f8a2800ce30 comp rx=0 tx=0).stop 2024-04-23T18:36:11.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a39bf7640 1 -- 172.21.15.132:0/2183023859 shutdown_connections 2024-04-23T18:36:11.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a39bf7640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8a1c0a3e40 0x7f8a1c0a6300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a39bf7640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a3419cfb0 0x7f8a3419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a39bf7640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a34107250 0x7f8a34198c80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a39bf7640 1 --2- 172.21.15.132:0/2183023859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a34102180 0x7f8a34198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a39bf7640 1 -- 172.21.15.132:0/2183023859 >> 172.21.15.132:0/2183023859 conn(0x7f8a340792d0 msgr2=0x7f8a34100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a39bf7640 1 -- 172.21.15.132:0/2183023859 shutdown_connections 2024-04-23T18:36:11.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.757+0000 7f8a39bf7640 1 -- 172.21.15.132:0/2183023859 wait complete. 2024-04-23T18:36:11.767 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-23T18:36:11.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:11 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/4050152766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:36:11.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:11 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/555365779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:11.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:11 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2796217688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:36:11.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.874+0000 7fa6394e8640 1 -- 172.21.15.132:0/1954208349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa634069350 msgr2=0x7fa634069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.874+0000 7fa6394e8640 1 --2- 172.21.15.132:0/1954208349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa634069350 0x7fa634069730 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa624009900 tx=0x7fa62402a410 comp rx=0 tx=0).stop 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.874+0000 7fa6394e8640 1 -- 172.21.15.132:0/1954208349 shutdown_connections 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.874+0000 7fa6394e8640 1 --2- 172.21.15.132:0/1954208349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa634107240 0x7fa63410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.874+0000 7fa6394e8640 1 --2- 172.21.15.132:0/1954208349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634069d00 0x7fa634106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.874+0000 7fa6394e8640 1 --2- 172.21.15.132:0/1954208349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa634069350 0x7fa634069730 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.874+0000 7fa6394e8640 1 -- 172.21.15.132:0/1954208349 >> 172.21.15.132:0/1954208349 conn(0x7fa634077cb0 msgr2=0x7fa6340780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.874+0000 7fa6394e8640 1 -- 172.21.15.132:0/1954208349 shutdown_connections 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.874+0000 7fa6394e8640 1 -- 172.21.15.132:0/1954208349 wait complete. 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa6394e8640 1 Processor -- start 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa6394e8640 1 -- start start 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa6394e8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa634069350 0x7fa6341986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa6394e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634107240 0x7fa634198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa6394e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa63419cf90 0x7fa63419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa6394e8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341121b0 con 0x7fa634069350 2024-04-23T18:36:11.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa6394e8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa634112030 con 0x7fa634107240 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa6394e8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa634112330 con 0x7fa63419cf90 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa632ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa634069350 0x7fa6341986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa632ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa634069350 0x7fa6341986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58678/0 (socket says 172.21.15.132:58678) 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa632ffd640 1 -- 172.21.15.132:0/1821686989 learned_addr learned my addr 172.21.15.132:0/1821686989 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa6327fc640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634107240 0x7fa634198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.875+0000 7fa6337fe640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa63419cf90 0x7fa63419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.876+0000 7fa632ffd640 1 -- 172.21.15.132:0/1821686989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa63419cf90 msgr2=0x7fa63419d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.876+0000 7fa632ffd640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa63419cf90 0x7fa63419d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.876+0000 7fa632ffd640 1 -- 172.21.15.132:0/1821686989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634107240 msgr2=0x7fa634198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.876+0000 7fa632ffd640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634107240 0x7fa634198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.876+0000 7fa632ffd640 1 -- 172.21.15.132:0/1821686989 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63419da80 con 0x7fa634069350 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.876+0000 7fa632ffd640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa634069350 0x7fa6341986d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa624005940 tx=0x7fa6240040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.876+0000 7fa61bfff640 1 -- 172.21.15.132:0/1821686989 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62403f030 con 0x7fa634069350 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.876+0000 7fa61bfff640 1 -- 172.21.15.132:0/1821686989 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6240044c0 con 0x7fa634069350 2024-04-23T18:36:11.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.876+0000 7fa6394e8640 1 -- 172.21.15.132:0/1821686989 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63419dd10 con 0x7fa634069350 2024-04-23T18:36:11.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.876+0000 7fa61bfff640 1 -- 172.21.15.132:0/1821686989 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624035070 con 0x7fa634069350 2024-04-23T18:36:11.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.876+0000 7fa6394e8640 1 -- 172.21.15.132:0/1821686989 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6341a54d0 con 0x7fa634069350 2024-04-23T18:36:11.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.878+0000 7fa61bfff640 1 -- 172.21.15.132:0/1821686989 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fa624042030 con 0x7fa634069350 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa63c1fe740 1 Processor -- start 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa63c1fe740 1 -- start start 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa61bfff640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa61c0a3e40 0x7fa61c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa63c1fe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c14db0 0x5622d8c15190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa63c1fe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c15760 0x5622d8c1f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa63c1fe740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 0x5622d8c22220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa63c1fe740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5622d8b7ac50 con 0x5622d8c1fdd0 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa63c1fe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5622d8b17830 con 0x5622d8c14db0 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa6337fe640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa61c0a3e40 0x7fa61c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa63c1fe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5622d8b76c10 con 0x5622d8c15760 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa61bfff640 1 -- 172.21.15.132:0/1821686989 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fa6240f0d30 con 0x7fa634069350 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 0x5622d8c22220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 0x5622d8c22220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58688/0 (socket says 172.21.15.132:58688) 2024-04-23T18:36:11.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa6337fe640 1 -- 172.21.15.132:0/2316862844 learned_addr learned my addr 172.21.15.132:0/2316862844 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa6327fc640 1 --2- 172.21.15.132:0/2316862844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c15760 0x5622d8c1f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa632ffd640 1 --2- 172.21.15.132:0/2316862844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c14db0 0x5622d8c15190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa6337fe640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa61c0a3e40 0x7fa61c0a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009fd0 tx=0x7fa62c013b80 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa6337fe640 1 -- 172.21.15.132:0/2316862844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c15760 msgr2=0x5622d8c1f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa6337fe640 1 --2- 172.21.15.132:0/2316862844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c15760 0x5622d8c1f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa6337fe640 1 -- 172.21.15.132:0/2316862844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c14db0 msgr2=0x5622d8c15190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa6337fe640 1 --2- 172.21.15.132:0/2316862844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c14db0 0x5622d8c15190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa6337fe640 1 -- 172.21.15.132:0/2316862844 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622d888d2d0 con 0x5622d8c1fdd0 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa632ffd640 1 --2- 172.21.15.132:0/2316862844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c14db0 0x5622d8c15190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.879+0000 7fa6337fe640 1 --2- 172.21.15.132:0/2316862844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 0x5622d8c22220 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00fc80 tx=0x7fa62c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc2440302f8a5d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.880+0000 7fa6197fa640 1 -- 172.21.15.132:0/2316862844 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0120d0 con 0x5622d8c1fdd0 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.880+0000 7fa6197fa640 1 -- 172.21.15.132:0/2316862844 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62c012270 con 0x5622d8c1fdd0 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.880+0000 7fa6197fa640 1 -- 172.21.15.132:0/2316862844 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c005280 con 0x5622d8c1fdd0 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.880+0000 7fa63c1fe740 1 -- 172.21.15.132:0/2316862844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 msgr2=0x5622d8c22220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.880+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/2316862844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 0x5622d8c22220 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00fc80 tx=0x7fa62c01f000 comp rx=0 tx=0).stop 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.880+0000 7fa63c1fe740 1 -- 172.21.15.132:0/2316862844 shutdown_connections 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.880+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/2316862844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c15760 0x5622d8c1f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.880+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/2316862844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 0x5622d8c22220 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.880+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/2316862844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c14db0 0x5622d8c15190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.880+0000 7fa63c1fe740 1 -- 172.21.15.132:0/2316862844 >> 172.21.15.132:0/2316862844 conn(0x5622d8c11070 msgr2=0x5622d8c11450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.880+0000 7fa63c1fe740 1 -- 172.21.15.132:0/2316862844 shutdown_connections 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.880+0000 7fa63c1fe740 1 -- 172.21.15.132:0/2316862844 wait complete. 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa63c1fe740 1 -- start start 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa63c1fe740 1 Processor -- start 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa63c1fe740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c14db0 0x5622d8c1cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa63c1fe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c15760 0x5622d8c1d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa63c1fe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c1fdd0 0x5622d8c1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa63c1fe740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5622d8c22c80 con 0x5622d8c14db0 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa63c1fe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5622d8b15470 con 0x5622d8c1fdd0 2024-04-23T18:36:11.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa63c1fe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5622d8c22e00 con 0x5622d8c15760 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa632ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c14db0 0x5622d8c1cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa63c1fe740 10 client.0 _ll_register_callbacks cb 0x5622d8b3af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa632ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c14db0 0x5622d8c1cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58694/0 (socket says 172.21.15.132:58694) 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa632ffd640 1 -- 172.21.15.132:0/3077523933 learned_addr learned my addr 172.21.15.132:0/3077523933 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa6337fe640 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c1fdd0 0x5622d8c1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa632ffd640 1 -- 172.21.15.132:0/3077523933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c15760 msgr2=0x5622d8c1d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa6327fc640 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c15760 0x5622d8c1d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa632ffd640 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c15760 0x5622d8c1d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa632ffd640 1 -- 172.21.15.132:0/3077523933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c1fdd0 msgr2=0x5622d8c1d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa632ffd640 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c1fdd0 0x5622d8c1d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa632ffd640 1 -- 172.21.15.132:0/3077523933 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622d89576b0 con 0x5622d8c14db0 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa6327fc640 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c15760 0x5622d8c1d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa632ffd640 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c14db0 0x5622d8c1cca0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa6240bc8f0 tx=0x7fa6240bc610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bd92180291da28 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.881+0000 7fa616ff5640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:11.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa616ff5640 1 -- 172.21.15.132:0/3077523933 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624049070 con 0x5622d8c14db0 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa616ff5640 1 -- 172.21.15.132:0/3077523933 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fa624003cf0 con 0x5622d8c14db0 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa63c1fe740 1 -- 172.21.15.132:0/3077523933 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa62c012270 con 0x5622d8c14db0 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa616ff5640 1 -- 172.21.15.132:0/3077523933 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa6240c57b0 con 0x5622d8c14db0 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa63c1fe740 10 client.15612 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa63c1fe740 1 -- 172.21.15.132:0/3077523933 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622d8c2ef40 con 0x5622d8c14db0 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa616ff5640 1 -- 172.21.15.132:0/3077523933 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62c012270 con 0x5622d8c14db0 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa616ff5640 1 -- 172.21.15.132:0/3077523933 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa6240bce50 con 0x5622d8c14db0 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa63c1fe740 10 client.15612 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa63c1fe740 10 client.15612 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa63c1fe740 20 client.15612 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x5622d8c31600 0x5622d8c32a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa63c1fe740 4 client.15612 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa63c1fe740 1 -- 172.21.15.132:0/3077523933 --> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622d8b88670 con 0x5622d8c31600 2024-04-23T18:36:11.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.882+0000 7fa6327fc640 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x5622d8c31600 0x5622d8c32a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.883+0000 7fa616ff5640 10 client.15612 ms_handle_connect on v2:172.21.15.148:6832/3556073608 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.883+0000 7fa6327fc640 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x5622d8c31600 0x5622d8c32a70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6ba9bd8c7b1166cb server_cookie=40b030415ba07ebd in_seq=0 out_seq=0 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.883+0000 7fa616ff5640 1 -- 172.21.15.132:0/3077523933 <== mds.3 v2:172.21.15.148:6832/3556073608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622d8b88670 con 0x5622d8c31600 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.883+0000 7fa616ff5640 10 client.15612 handle_command_reply: tid=0 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.883+0000 7fa63c1fe740 1 client.15612 shutdown 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.884+0000 7fa63c1fe740 1 -- 172.21.15.132:0/3077523933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c14db0 msgr2=0x5622d8c1cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.884+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c14db0 0x5622d8c1cca0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa6240bc8f0 tx=0x7fa6240bc610 comp rx=0 tx=0).stop 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.884+0000 7fa63c1fe740 1 -- 172.21.15.132:0/3077523933 shutdown_connections 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.884+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x5622d8c31600 0x5622d8c32a70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.884+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c15760 0x5622d8c1d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.884+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c1fdd0 0x5622d8c1d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.884+0000 7fa616ff5640 0 client.15612 ms_handle_reset on v2:172.21.15.148:6832/3556073608 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.884+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/3077523933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c14db0 0x5622d8c1cca0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.884+0000 7fa63c1fe740 1 -- 172.21.15.132:0/3077523933 >> 172.21.15.132:0/3077523933 conn(0x5622d8c11070 msgr2=0x5622d8c16ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.884+0000 7fa63c1fe740 1 -- 172.21.15.132:0/3077523933 shutdown_connections 2024-04-23T18:36:11.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.884+0000 7fa63c1fe740 1 -- 172.21.15.132:0/3077523933 wait complete. 2024-04-23T18:36:11.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.884+0000 7fa63c1fe740 20 client.15612 trim_cache size 0 max 16384 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.901+0000 7fa63c1fe740 1 Processor -- start 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.901+0000 7fa63c1fe740 1 -- start start 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa63c1fe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c14db0 0x5622d8c17520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa63c1fe740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c15760 0x5622d8c1b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa63c1fe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c1fdd0 0x5622d8c1bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa63c1fe740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5622d8c30db0 con 0x5622d8c15760 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa63c1fe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5622d8c31480 con 0x5622d8c1fdd0 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa63c1fe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5622d8c31b50 con 0x5622d8c14db0 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa6327fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c15760 0x5622d8c1b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa6327fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c15760 0x5622d8c1b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58698/0 (socket says 172.21.15.132:58698) 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa6327fc640 1 -- 172.21.15.132:0/1338550231 learned_addr learned my addr 172.21.15.132:0/1338550231 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa6327fc640 1 -- 172.21.15.132:0/1338550231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c14db0 msgr2=0x5622d8c17520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa6327fc640 1 --2- 172.21.15.132:0/1338550231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c14db0 0x5622d8c17520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa6327fc640 1 -- 172.21.15.132:0/1338550231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c1fdd0 msgr2=0x5622d8c1bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa6327fc640 1 --2- 172.21.15.132:0/1338550231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c1fdd0 0x5622d8c1bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa6327fc640 1 -- 172.21.15.132:0/1338550231 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622d8c30970 con 0x5622d8c15760 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa6327fc640 1 --2- 172.21.15.132:0/1338550231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c15760 0x5622d8c1b570 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x5622d8c16370 tx=0x7fa628004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f4c328ce9ebc00 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa618ff9640 1 -- 172.21.15.132:0/1338550231 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628021020 con 0x5622d8c15760 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa618ff9640 1 -- 172.21.15.132:0/1338550231 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62801d880 con 0x5622d8c15760 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.902+0000 7fa618ff9640 1 -- 172.21.15.132:0/1338550231 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628010960 con 0x5622d8c15760 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 -- 172.21.15.132:0/1338550231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c15760 msgr2=0x5622d8c1b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/1338550231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c15760 0x5622d8c1b570 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x5622d8c16370 tx=0x7fa628004410 comp rx=0 tx=0).stop 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 -- 172.21.15.132:0/1338550231 shutdown_connections 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/1338550231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c1fdd0 0x5622d8c1bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/1338550231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c15760 0x5622d8c1b570 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/1338550231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c14db0 0x5622d8c17520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 -- 172.21.15.132:0/1338550231 >> 172.21.15.132:0/1338550231 conn(0x5622d8c11070 msgr2=0x5622d8c11450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 -- 172.21.15.132:0/1338550231 shutdown_connections 2024-04-23T18:36:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 -- 172.21.15.132:0/1338550231 wait complete. 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 -- start start 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 Processor -- start 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c14db0 0x5622d8c347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c15760 0x5622d8c34ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 0x5622d8c35220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5622d8c328f0 con 0x5622d8c1fdd0 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5622d8c32220 con 0x5622d8c15760 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5622d8c32fc0 con 0x5622d8c14db0 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.903+0000 7fa63c1fe740 10 client.0 _ll_register_callbacks cb 0x7fa624005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 0x5622d8c35220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 0x5622d8c35220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58700/0 (socket says 172.21.15.132:58700) 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa6337fe640 1 -- 172.21.15.132:0/4274601950 learned_addr learned my addr 172.21.15.132:0/4274601950 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa632ffd640 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c14db0 0x5622d8c347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa6337fe640 1 -- 172.21.15.132:0/4274601950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c14db0 msgr2=0x5622d8c347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa6337fe640 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c14db0 0x5622d8c347a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa6337fe640 1 -- 172.21.15.132:0/4274601950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c15760 msgr2=0x5622d8c34ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa6337fe640 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c15760 0x5622d8c34ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa6337fe640 1 -- 172.21.15.132:0/4274601950 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622d8c38fe0 con 0x5622d8c1fdd0 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa632ffd640 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c14db0 0x5622d8c347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa6337fe640 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 0x5622d8c35220 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00f830 tx=0x7fa62c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56725543feafd0ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa616ff5640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa616ff5640 1 -- 172.21.15.132:0/4274601950 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c02f060 con 0x5622d8c1fdd0 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa616ff5640 1 -- 172.21.15.132:0/4274601950 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fa62c005050 con 0x5622d8c1fdd0 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa63c1fe740 1 -- 172.21.15.132:0/4274601950 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa62801d880 con 0x5622d8c1fdd0 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa616ff5640 1 -- 172.21.15.132:0/4274601950 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa62c009c30 con 0x5622d8c1fdd0 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa63c1fe740 10 client.15618 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa63c1fe740 1 -- 172.21.15.132:0/4274601950 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622d8c978e0 con 0x5622d8c1fdd0 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.904+0000 7fa616ff5640 1 -- 172.21.15.132:0/4274601950 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62801d880 con 0x5622d8c1fdd0 2024-04-23T18:36:11.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa616ff5640 1 -- 172.21.15.132:0/4274601950 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa62c020780 con 0x5622d8c1fdd0 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa63c1fe740 10 client.15618 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa63c1fe740 10 client.15618 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa63c1fe740 20 client.15618 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x5622d8c9c200 0x5622d8c9af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa63c1fe740 4 client.15618 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa63c1fe740 1 -- 172.21.15.132:0/4274601950 --> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] -- command(tid 0: {"prefix": "status"}) -- 0x5622d8b824e0 con 0x5622d8c9c200 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa632ffd640 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x5622d8c9c200 0x5622d8c9af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa632ffd640 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x5622d8c9c200 0x5622d8c9af10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bcfb86cc56b26ba7 server_cookie=868be4387a3b3763 in_seq=0 out_seq=0 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa616ff5640 10 client.15618 ms_handle_connect on v2:172.21.15.148:6832/3556073608 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa616ff5640 1 -- 172.21.15.132:0/4274601950 <== mds.3 v2:172.21.15.148:6832/3556073608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5622d8b824e0 con 0x5622d8c9c200 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa616ff5640 10 client.15618 handle_command_reply: tid=0 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa63c1fe740 1 client.15618 shutdown 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa63c1fe740 1 -- 172.21.15.132:0/4274601950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 msgr2=0x5622d8c35220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.905+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 0x5622d8c35220 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00f830 tx=0x7fa62c00fab0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa63c1fe740 1 -- 172.21.15.132:0/4274601950 shutdown_connections 2024-04-23T18:36:11.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.148:6832/3556073608,v1:172.21.15.148:6833/3556073608] conn(0x5622d8c9c200 0x5622d8c9af10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622d8c1fdd0 0x5622d8c35220 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa616ff5640 0 client.15618 ms_handle_reset on v2:172.21.15.148:6832/3556073608 2024-04-23T18:36:11.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622d8c15760 0x5622d8c34ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa63c1fe740 1 --2- 172.21.15.132:0/4274601950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622d8c14db0 0x5622d8c347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa63c1fe740 1 -- 172.21.15.132:0/4274601950 >> 172.21.15.132:0/4274601950 conn(0x5622d8c11070 msgr2=0x5622d8c1b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa63c1fe740 1 -- 172.21.15.132:0/4274601950 shutdown_connections 2024-04-23T18:36:11.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa63c1fe740 1 -- 172.21.15.132:0/4274601950 wait complete. 2024-04-23T18:36:11.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa63c1fe740 20 client.15618 trim_cache size 0 max 16384 2024-04-23T18:36:11.904 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:36:11.904 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:36:11.905 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 3, 2024-04-23T18:36:11.905 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24425, 2024-04-23T18:36:11.905 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:36:11.905 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:36:11.905 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:36:11.905 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 126.384727674, 2024-04-23T18:36:11.905 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 26, 2024-04-23T18:36:11.905 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 91, 2024-04-23T18:36:11.905 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 90, 2024-04-23T18:36:11.905 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 166.27141019699999 2024-04-23T18:36:11.905 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:36:11.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa6394e8640 1 -- 172.21.15.132:0/1821686989 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa61c0a3e40 msgr2=0x7fa61c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa6394e8640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa61c0a3e40 0x7fa61c0a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009fd0 tx=0x7fa62c013b80 comp rx=0 tx=0).stop 2024-04-23T18:36:11.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa6394e8640 1 -- 172.21.15.132:0/1821686989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa634069350 msgr2=0x7fa6341986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:11.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa6394e8640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa634069350 0x7fa6341986d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa624005940 tx=0x7fa6240040c0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa6394e8640 1 -- 172.21.15.132:0/1821686989 shutdown_connections 2024-04-23T18:36:11.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa6394e8640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fa61c0a3e40 0x7fa61c0a6300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa6394e8640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa63419cf90 0x7fa63419d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa6394e8640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634107240 0x7fa634198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa6394e8640 1 --2- 172.21.15.132:0/1821686989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa634069350 0x7fa6341986d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:11.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa6394e8640 1 -- 172.21.15.132:0/1821686989 >> 172.21.15.132:0/1821686989 conn(0x7fa634077cb0 msgr2=0x7fa6340ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:11.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa6394e8640 1 -- 172.21.15.132:0/1821686989 shutdown_connections 2024-04-23T18:36:11.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:11.906+0000 7fa6394e8640 1 -- 172.21.15.132:0/1821686989 wait complete. 2024-04-23T18:36:11.910 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T18:36:12.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.017+0000 7f1d382e1640 1 -- 172.21.15.132:0/3298600145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d301010e0 msgr2=0x7f1d30101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.017+0000 7f1d382e1640 1 --2- 172.21.15.132:0/3298600145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d301010e0 0x7f1d30101560 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009900 tx=0x7f1d2002a470 comp rx=0 tx=0).stop 2024-04-23T18:36:12.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.017+0000 7f1d382e1640 1 -- 172.21.15.132:0/3298600145 shutdown_connections 2024-04-23T18:36:12.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.017+0000 7f1d382e1640 1 --2- 172.21.15.132:0/3298600145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d30101aa0 0x7f1d3010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.017+0000 7f1d382e1640 1 --2- 172.21.15.132:0/3298600145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d301010e0 0x7f1d30101560 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.017+0000 7f1d382e1640 1 --2- 172.21.15.132:0/3298600145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d30105e70 0x7f1d30106250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.017+0000 7f1d382e1640 1 -- 172.21.15.132:0/3298600145 >> 172.21.15.132:0/3298600145 conn(0x7f1d30077cb0 msgr2=0x7f1d300780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d382e1640 1 -- 172.21.15.132:0/3298600145 shutdown_connections 2024-04-23T18:36:12.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d382e1640 1 -- 172.21.15.132:0/3298600145 wait complete. 2024-04-23T18:36:12.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d382e1640 1 Processor -- start 2024-04-23T18:36:12.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d382e1640 1 -- start start 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d382e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d301010e0 0x7f1d30198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d382e1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d30101aa0 0x7f1d30198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d382e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d30105e70 0x7f1d3019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d382e1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301110f0 con 0x7f1d30101aa0 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d382e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30110f70 con 0x7f1d301010e0 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d382e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30111270 con 0x7f1d30105e70 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d3587c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d30101aa0 0x7f1d30198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d3587c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d30101aa0 0x7f1d30198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58716/0 (socket says 172.21.15.132:58716) 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.018+0000 7f1d3587c640 1 -- 172.21.15.132:0/2717905065 learned_addr learned my addr 172.21.15.132:0/2717905065 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d3687e640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d30105e70 0x7f1d3019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d3607d640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d301010e0 0x7f1d30198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d3587c640 1 -- 172.21.15.132:0/2717905065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d30105e70 msgr2=0x7f1d3019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d3587c640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d30105e70 0x7f1d3019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d3587c640 1 -- 172.21.15.132:0/2717905065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d301010e0 msgr2=0x7f1d30198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d3587c640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d301010e0 0x7f1d30198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d3587c640 1 -- 172.21.15.132:0/2717905065 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3019c6d0 con 0x7f1d30101aa0 2024-04-23T18:36:12.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d3607d640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d301010e0 0x7f1d30198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:12.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d3587c640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d30101aa0 0x7f1d30198cd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1d20006010 tx=0x7f1d200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d277fe640 1 -- 172.21.15.132:0/2717905065 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2003f030 con 0x7f1d30101aa0 2024-04-23T18:36:12.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d382e1640 1 -- 172.21.15.132:0/2717905065 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3019c960 con 0x7f1d30101aa0 2024-04-23T18:36:12.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d277fe640 1 -- 172.21.15.132:0/2717905065 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d20003cb0 con 0x7f1d30101aa0 2024-04-23T18:36:12.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d277fe640 1 -- 172.21.15.132:0/2717905065 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20035070 con 0x7f1d30101aa0 2024-04-23T18:36:12.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.019+0000 7f1d382e1640 1 -- 172.21.15.132:0/2717905065 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d301a5330 con 0x7f1d30101aa0 2024-04-23T18:36:12.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:11 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/4050152766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:36:12.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:11 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/555365779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:12.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:11 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2796217688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:36:12.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.022+0000 7f1d277fe640 1 -- 172.21.15.132:0/2717905065 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f1d20042030 con 0x7f1d30101aa0 2024-04-23T18:36:12.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.022+0000 7f1d3aff9740 1 Processor -- start 2024-04-23T18:36:12.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.022+0000 7f1d3aff9740 1 -- start start 2024-04-23T18:36:12.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.022+0000 7f1d3aff9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 0x556eaa2c9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.022+0000 7f1d3aff9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 0x556eaa2d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3aff9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2d4210 0x556eaa2d4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3aff9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x556eaa22ec50 con 0x556eaa2c95e0 2024-04-23T18:36:12.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3aff9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556eaa1cb830 con 0x556eaa2d4210 2024-04-23T18:36:12.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3aff9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556eaa22ac10 con 0x556eaa2c8c30 2024-04-23T18:36:12.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3607d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 0x556eaa2d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3607d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 0x556eaa2d3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58728/0 (socket says 172.21.15.132:58728) 2024-04-23T18:36:12.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3607d640 1 -- 172.21.15.132:0/3438570250 learned_addr learned my addr 172.21.15.132:0/3438570250 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3687e640 1 --2- 172.21.15.132:0/3438570250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 0x556eaa2c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d277fe640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1d1c0a3e40 0x7f1d1c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3607d640 1 -- 172.21.15.132:0/3438570250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 msgr2=0x556eaa2c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3587c640 1 --2- 172.21.15.132:0/3438570250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2d4210 0x556eaa2d4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3607d640 1 --2- 172.21.15.132:0/3438570250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 0x556eaa2c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3607d640 1 -- 172.21.15.132:0/3438570250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2d4210 msgr2=0x556eaa2d4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3687e640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1d1c0a3e40 0x7f1d1c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3607d640 1 --2- 172.21.15.132:0/3438570250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2d4210 0x556eaa2d4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3607d640 1 -- 172.21.15.132:0/3438570250 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ea9f412d0 con 0x556eaa2c95e0 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d277fe640 1 -- 172.21.15.132:0/2717905065 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f1d200f0ed0 con 0x7f1d30101aa0 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3587c640 1 --2- 172.21.15.132:0/3438570250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2d4210 0x556eaa2d4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3607d640 1 --2- 172.21.15.132:0/3438570250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 0x556eaa2d3bf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009e90 tx=0x7f1d2c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0307e27becf22c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d3687e640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1d1c0a3e40 0x7f1d1c0a6300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1d28005850 tx=0x7f1d28004730 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d24bfe640 1 -- 172.21.15.132:0/3438570250 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c01a680 con 0x556eaa2c95e0 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d24bfe640 1 -- 172.21.15.132:0/3438570250 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d2c0118f0 con 0x556eaa2c95e0 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.023+0000 7f1d24bfe640 1 -- 172.21.15.132:0/3438570250 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0107f0 con 0x556eaa2c95e0 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.024+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3438570250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 msgr2=0x556eaa2d3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.024+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/3438570250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 0x556eaa2d3bf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009e90 tx=0x7f1d2c00f730 comp rx=0 tx=0).stop 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.024+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3438570250 shutdown_connections 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.024+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/3438570250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2d4210 0x556eaa2d4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.024+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/3438570250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 0x556eaa2d3bf0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.024+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/3438570250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 0x556eaa2c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.024+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3438570250 >> 172.21.15.132:0/3438570250 conn(0x556eaa2c5070 msgr2=0x556eaa2c5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.024+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3438570250 shutdown_connections 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.024+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3438570250 wait complete. 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3aff9740 1 -- start start 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3aff9740 1 Processor -- start 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3aff9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c8c30 0x556eaa2d7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3aff9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c95e0 0x556eaa2d80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3aff9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2d4210 0x556eaa2d85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3aff9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x556eaa2db190 con 0x556eaa2c8c30 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3aff9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556eaa1c9470 con 0x556eaa2c95e0 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3aff9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556eaa2db310 con 0x556eaa2d4210 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3aff9740 10 client.0 _ll_register_callbacks cb 0x556eaa1eef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3607d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c8c30 0x556eaa2d7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3607d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c8c30 0x556eaa2d7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58742/0 (socket says 172.21.15.132:58742) 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3607d640 1 -- 172.21.15.132:0/3380659143 learned_addr learned my addr 172.21.15.132:0/3380659143 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3687e640 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2d4210 0x556eaa2d85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3607d640 1 -- 172.21.15.132:0/3380659143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2d4210 msgr2=0x556eaa2d85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3587c640 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c95e0 0x556eaa2d80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3607d640 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2d4210 0x556eaa2d85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3607d640 1 -- 172.21.15.132:0/3380659143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c95e0 msgr2=0x556eaa2d80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3607d640 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c95e0 0x556eaa2d80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3607d640 1 -- 172.21.15.132:0/3380659143 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eaa00b6b0 con 0x556eaa2c8c30 2024-04-23T18:36:12.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d3607d640 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c8c30 0x556eaa2d7b60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c012fd0 tx=0x7f1d2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fefcca6f1375ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.025+0000 7f1d19ffb640 1 -- 172.21.15.132:0/3380659143 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c029060 con 0x556eaa2c8c30 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d19ffb640 1 -- 172.21.15.132:0/3380659143 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f1d2c011aa0 con 0x556eaa2c8c30 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3380659143 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d2c0118f0 con 0x556eaa2c8c30 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d19ffb640 1 -- 172.21.15.132:0/3380659143 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d2c011ca0 con 0x556eaa2c8c30 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d3aff9740 10 client.15630 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3380659143 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556eaa2e7310 con 0x556eaa2c8c30 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d19ffb640 1 -- 172.21.15.132:0/3380659143 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d2c0118f0 con 0x556eaa2c8c30 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d19ffb640 1 -- 172.21.15.132:0/3380659143 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1d2c010bc0 con 0x556eaa2c8c30 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d3aff9740 10 client.15630 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d3aff9740 10 client.15630 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d3aff9740 20 client.15630 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x556eaa2e94a0 0x556eaa2ea910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d3aff9740 4 client.15630 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3380659143 --> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eaa23c670 con 0x556eaa2e94a0 2024-04-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.026+0000 7f1d3587c640 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x556eaa2e94a0 0x556eaa2ea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.027+0000 7f1d3587c640 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x556eaa2e94a0 0x556eaa2ea910 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3fc428f44fa9b8e2 server_cookie=603850962185b84d in_seq=0 out_seq=0 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.027+0000 7f1d19ffb640 10 client.15630 ms_handle_connect on v2:172.21.15.157:6838/1528717147 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.027+0000 7f1d19ffb640 1 -- 172.21.15.132:0/3380659143 <== mds.2 v2:172.21.15.157:6838/1528717147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556eaa23c670 con 0x556eaa2e94a0 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.027+0000 7f1d19ffb640 10 client.15630 handle_command_reply: tid=0 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.027+0000 7f1d3aff9740 1 client.15630 shutdown 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.028+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3380659143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c8c30 msgr2=0x556eaa2d7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.028+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c8c30 0x556eaa2d7b60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c012fd0 tx=0x7f1d2c009bd0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.028+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3380659143 shutdown_connections 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.028+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x556eaa2e94a0 0x556eaa2ea910 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.028+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2d4210 0x556eaa2d85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.028+0000 7f1d19ffb640 0 client.15630 ms_handle_reset on v2:172.21.15.157:6838/1528717147 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.028+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c95e0 0x556eaa2d80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.028+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/3380659143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c8c30 0x556eaa2d7b60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.028+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3380659143 >> 172.21.15.132:0/3380659143 conn(0x556eaa2c5070 msgr2=0x556eaa2caee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.028+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3380659143 shutdown_connections 2024-04-23T18:36:12.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.028+0000 7f1d3aff9740 1 -- 172.21.15.132:0/3380659143 wait complete. 2024-04-23T18:36:12.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.028+0000 7f1d3aff9740 20 client.15630 trim_cache size 0 max 16384 2024-04-23T18:36:12.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:11 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/4050152766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:36:12.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:11 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/555365779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:12.043 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:11 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2796217688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:36:12.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.045+0000 7f1d3aff9740 1 Processor -- start 2024-04-23T18:36:12.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.045+0000 7f1d3aff9740 1 -- start start 2024-04-23T18:36:12.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3aff9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c5070 0x556eaa2da290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3aff9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 0x556eaa2d5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3aff9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c95e0 0x556eaa2d8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3aff9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x556eaa2e8c50 con 0x556eaa2c5070 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3aff9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556eaa2e9320 con 0x556eaa2c95e0 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3aff9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556eaa2e99f0 con 0x556eaa2c8c30 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3607d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c5070 0x556eaa2da290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3607d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c5070 0x556eaa2da290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58756/0 (socket says 172.21.15.132:58756) 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3607d640 1 -- 172.21.15.132:0/2258504504 learned_addr learned my addr 172.21.15.132:0/2258504504 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3607d640 1 -- 172.21.15.132:0/2258504504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 msgr2=0x556eaa2d5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3607d640 1 --2- 172.21.15.132:0/2258504504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 0x556eaa2d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3607d640 1 -- 172.21.15.132:0/2258504504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c95e0 msgr2=0x556eaa2d8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3607d640 1 --2- 172.21.15.132:0/2258504504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c95e0 0x556eaa2d8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3607d640 1 -- 172.21.15.132:0/2258504504 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eaa2d5b40 con 0x556eaa2c5070 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d3607d640 1 --2- 172.21.15.132:0/2258504504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c5070 0x556eaa2da290 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c002230 tx=0x7f1d2c005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4da9d0ec596819 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d1bfff640 1 -- 172.21.15.132:0/2258504504 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c01e020 con 0x556eaa2c5070 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d1bfff640 1 -- 172.21.15.132:0/2258504504 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d2c011890 con 0x556eaa2c5070 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.046+0000 7f1d1bfff640 1 -- 172.21.15.132:0/2258504504 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c029040 con 0x556eaa2c5070 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 -- 172.21.15.132:0/2258504504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c5070 msgr2=0x556eaa2da290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/2258504504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c5070 0x556eaa2da290 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c002230 tx=0x7f1d2c005440 comp rx=0 tx=0).stop 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 -- 172.21.15.132:0/2258504504 shutdown_connections 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/2258504504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 0x556eaa2d5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/2258504504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c95e0 0x556eaa2d8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/2258504504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c5070 0x556eaa2da290 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 -- 172.21.15.132:0/2258504504 >> 172.21.15.132:0/2258504504 conn(0x556eaa2d2120 msgr2=0x556eaa2bfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 -- 172.21.15.132:0/2258504504 shutdown_connections 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 -- 172.21.15.132:0/2258504504 wait complete. 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 -- start start 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 Processor -- start 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c5070 0x556eaa2f3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 0x556eaa2d7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 0x556eaa2d8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x556eaa2ea790 con 0x556eaa2c95e0 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556eaa2ea0c0 con 0x556eaa2c5070 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556eaa2eae60 con 0x556eaa2c8c30 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3aff9740 10 client.0 _ll_register_callbacks cb 0x7f1d2800b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3687e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 0x556eaa2d8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3687e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 0x556eaa2d8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58768/0 (socket says 172.21.15.132:58768) 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.047+0000 7f1d3687e640 1 -- 172.21.15.132:0/1863587360 learned_addr learned my addr 172.21.15.132:0/1863587360 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d3607d640 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c5070 0x556eaa2f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d3587c640 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 0x556eaa2d7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d3687e640 1 -- 172.21.15.132:0/1863587360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 msgr2=0x556eaa2d7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d3687e640 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 0x556eaa2d7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d3687e640 1 -- 172.21.15.132:0/1863587360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c5070 msgr2=0x556eaa2f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d3687e640 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c5070 0x556eaa2f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d3687e640 1 -- 172.21.15.132:0/1863587360 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eaa2d8850 con 0x556eaa2c95e0 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d3687e640 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 0x556eaa2d8080 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1d280107a0 tx=0x7f1d2800ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7a97fac337777f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d19ffb640 1 -- 172.21.15.132:0/1863587360 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28019070 con 0x556eaa2c95e0 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d3aff9740 1 -- 172.21.15.132:0/1863587360 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d2c011890 con 0x556eaa2c95e0 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d19ffb640 1 -- 172.21.15.132:0/1863587360 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f1d28005220 con 0x556eaa2c95e0 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d19ffb640 1 -- 172.21.15.132:0/1863587360 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d28005420 con 0x556eaa2c95e0 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d3aff9740 10 client.15636 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d3aff9740 1 -- 172.21.15.132:0/1863587360 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556eaa34f5a0 con 0x556eaa2c95e0 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d19ffb640 1 -- 172.21.15.132:0/1863587360 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d2c011890 con 0x556eaa2c95e0 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d19ffb640 1 -- 172.21.15.132:0/1863587360 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1d2801c070 con 0x556eaa2c95e0 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.048+0000 7f1d3aff9740 10 client.15636 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d3aff9740 10 client.15636 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d3aff9740 20 client.15636 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:12.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x556eaa2d4210 0x556eaa352e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d3aff9740 4 client.15636 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d3aff9740 1 -- 172.21.15.132:0/1863587360 --> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] -- command(tid 0: {"prefix": "status"}) -- 0x556eaa2364e0 con 0x556eaa2d4210 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d3607d640 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x556eaa2d4210 0x556eaa352e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d3607d640 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x556eaa2d4210 0x556eaa352e70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa6493d138b96962 server_cookie=3b9a9627d586a000 in_seq=0 out_seq=0 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d19ffb640 10 client.15636 ms_handle_connect on v2:172.21.15.157:6838/1528717147 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d19ffb640 1 -- 172.21.15.132:0/1863587360 <== mds.2 v2:172.21.15.157:6838/1528717147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556eaa2364e0 con 0x556eaa2d4210 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d19ffb640 10 client.15636 handle_command_reply: tid=0 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d3aff9740 1 client.15636 shutdown 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d3aff9740 1 -- 172.21.15.132:0/1863587360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 msgr2=0x556eaa2d8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.049+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 0x556eaa2d8080 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1d280107a0 tx=0x7f1d2800ba70 comp rx=0 tx=0).stop 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d3aff9740 1 -- 172.21.15.132:0/1863587360 shutdown_connections 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.157:6838/1528717147,v1:172.21.15.157:6839/1528717147] conn(0x556eaa2d4210 0x556eaa352e70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556eaa2c8c30 0x556eaa2d7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556eaa2c95e0 0x556eaa2d8080 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d3aff9740 1 --2- 172.21.15.132:0/1863587360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556eaa2c5070 0x556eaa2f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d3aff9740 1 -- 172.21.15.132:0/1863587360 >> 172.21.15.132:0/1863587360 conn(0x556eaa2d2120 msgr2=0x556eaa2d63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d19ffb640 0 client.15636 ms_handle_reset on v2:172.21.15.157:6838/1528717147 2024-04-23T18:36:12.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d3aff9740 1 -- 172.21.15.132:0/1863587360 shutdown_connections 2024-04-23T18:36:12.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d3aff9740 1 -- 172.21.15.132:0/1863587360 wait complete. 2024-04-23T18:36:12.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d3aff9740 20 client.15636 trim_cache size 0 max 16384 2024-04-23T18:36:12.049 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:36:12.049 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:36:12.049 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 2, 2024-04-23T18:36:12.049 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24460, 2024-04-23T18:36:12.049 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:36:12.050 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:36:12.050 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:36:12.050 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 127.50996350699999, 2024-04-23T18:36:12.050 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 26, 2024-04-23T18:36:12.050 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 91, 2024-04-23T18:36:12.050 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 90, 2024-04-23T18:36:12.050 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 146.210150301 2024-04-23T18:36:12.050 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:36:12.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d382e1640 1 -- 172.21.15.132:0/2717905065 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1d1c0a3e40 msgr2=0x7f1d1c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d382e1640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1d1c0a3e40 0x7f1d1c0a6300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1d28005850 tx=0x7f1d28004730 comp rx=0 tx=0).stop 2024-04-23T18:36:12.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d382e1640 1 -- 172.21.15.132:0/2717905065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d30101aa0 msgr2=0x7f1d30198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d382e1640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d30101aa0 0x7f1d30198cd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1d20006010 tx=0x7f1d200042f0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d382e1640 1 -- 172.21.15.132:0/2717905065 shutdown_connections 2024-04-23T18:36:12.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d382e1640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1d1c0a3e40 0x7f1d1c0a6300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d382e1640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d30105e70 0x7f1d3019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d382e1640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d30101aa0 0x7f1d30198cd0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d382e1640 1 --2- 172.21.15.132:0/2717905065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d301010e0 0x7f1d30198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d382e1640 1 -- 172.21.15.132:0/2717905065 >> 172.21.15.132:0/2717905065 conn(0x7f1d30077cb0 msgr2=0x7f1d300ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d382e1640 1 -- 172.21.15.132:0/2717905065 shutdown_connections 2024-04-23T18:36:12.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.050+0000 7f1d382e1640 1 -- 172.21.15.132:0/2717905065 wait complete. 2024-04-23T18:36:12.053 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T18:36:12.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.160+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/1277862257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64106200 msgr2=0x7f1d6410ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.160+0000 7f1d6ab0f640 1 --2- 172.21.15.132:0/1277862257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64106200 0x7f1d6410ddd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009ca0 tx=0x7f1d5c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.160+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/1277862257 shutdown_connections 2024-04-23T18:36:12.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.160+0000 7f1d6ab0f640 1 --2- 172.21.15.132:0/1277862257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d64101ae0 0x7f1d64105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.160+0000 7f1d6ab0f640 1 --2- 172.21.15.132:0/1277862257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64106200 0x7f1d6410ddd0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.160+0000 7f1d6ab0f640 1 --2- 172.21.15.132:0/1277862257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d64101130 0x7f1d64101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.160+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/1277862257 >> 172.21.15.132:0/1277862257 conn(0x7f1d64077f80 msgr2=0x7f1d64078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.160+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/1277862257 shutdown_connections 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.160+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/1277862257 wait complete. 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d6ab0f640 1 Processor -- start 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d6ab0f640 1 -- start start 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d6ab0f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64101130 0x7f1d641986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d6ab0f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d64106200 0x7f1d64198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d6ab0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d6419cfa0 0x7f1d6419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d6ab0f640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641111a0 con 0x7f1d64101130 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d6ab0f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64111020 con 0x7f1d6419cfa0 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d6ab0f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64111320 con 0x7f1d64106200 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d688ab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64101130 0x7f1d641986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d688ab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64101130 0x7f1d641986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58786/0 (socket says 172.21.15.132:58786) 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d688ab640 1 -- 172.21.15.132:0/636668757 learned_addr learned my addr 172.21.15.132:0/636668757 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d690ac640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d6419cfa0 0x7f1d6419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d688ab640 1 -- 172.21.15.132:0/636668757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d64106200 msgr2=0x7f1d64198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d688ab640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d64106200 0x7f1d64198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d688ab640 1 -- 172.21.15.132:0/636668757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d6419cfa0 msgr2=0x7f1d6419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d688ab640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d6419cfa0 0x7f1d6419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d688ab640 1 -- 172.21.15.132:0/636668757 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6419da90 con 0x7f1d64101130 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d690ac640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d6419cfa0 0x7f1d6419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d688ab640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64101130 0x7f1d641986e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800e9f0 tx=0x7f1d5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.161+0000 7f1d61ffb640 1 -- 172.21.15.132:0/636668757 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800cd80 con 0x7f1d64101130 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.162+0000 7f1d61ffb640 1 -- 172.21.15.132:0/636668757 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d580108e0 con 0x7f1d64101130 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.162+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/636668757 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6419dd80 con 0x7f1d64101130 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.162+0000 7f1d61ffb640 1 -- 172.21.15.132:0/636668757 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58018980 con 0x7f1d64101130 2024-04-23T18:36:12.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.162+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/636668757 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d641a5570 con 0x7f1d64101130 2024-04-23T18:36:12.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.164+0000 7f1d61ffb640 1 -- 172.21.15.132:0/636668757 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f1d58010430 con 0x7f1d64101130 2024-04-23T18:36:12.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.164+0000 7f1d6d821740 1 Processor -- start 2024-04-23T18:36:12.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.164+0000 7f1d6d821740 1 -- start start 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.164+0000 7f1d6d821740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6c2db0 0x55943c6c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d6d821740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c3760 0x55943c6cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d6d821740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6cddd0 0x55943c6d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d61ffb640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1d540a3e40 0x7f1d540a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d6d821740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55943c628c50 con 0x55943c6c3760 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d6d821740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55943c5c5830 con 0x55943c6c2db0 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d6d821740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55943c624c10 con 0x55943c6cddd0 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d688ab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c3760 0x55943c6cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d690ac640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1d540a3e40 0x7f1d540a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d688ab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c3760 0x55943c6cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58802/0 (socket says 172.21.15.132:58802) 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d688ab640 1 -- 172.21.15.132:0/2006120091 learned_addr learned my addr 172.21.15.132:0/2006120091 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d61ffb640 1 -- 172.21.15.132:0/636668757 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f1d580d0f00 con 0x7f1d64101130 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d63fff640 1 --2- 172.21.15.132:0/2006120091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6cddd0 0x55943c6d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d690ac640 1 --2- 172.21.15.132:0/2006120091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6c2db0 0x55943c6c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d688ab640 1 -- 172.21.15.132:0/2006120091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6cddd0 msgr2=0x55943c6d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d688ab640 1 --2- 172.21.15.132:0/2006120091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6cddd0 0x55943c6d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d688ab640 1 -- 172.21.15.132:0/2006120091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6c2db0 msgr2=0x55943c6c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d688ab640 1 --2- 172.21.15.132:0/2006120091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6c2db0 0x55943c6c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d688ab640 1 -- 172.21.15.132:0/2006120091 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55943c33b2d0 con 0x55943c6c3760 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d690ac640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1d540a3e40 0x7f1d540a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c02f540 tx=0x7f1d5c032d00 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d688ab640 1 --2- 172.21.15.132:0/2006120091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c3760 0x55943c6cd800 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1d580cec40 tx=0x7f1d5806b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e93f36eeca951 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d4effd640 1 -- 172.21.15.132:0/2006120091 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5809d340 con 0x55943c6c3760 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d4effd640 1 -- 172.21.15.132:0/2006120091 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d58087040 con 0x55943c6c3760 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.165+0000 7f1d4effd640 1 -- 172.21.15.132:0/2006120091 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58010a90 con 0x55943c6c3760 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.166+0000 7f1d6d821740 1 -- 172.21.15.132:0/2006120091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c3760 msgr2=0x55943c6cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.166+0000 7f1d6d821740 1 --2- 172.21.15.132:0/2006120091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c3760 0x55943c6cd800 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1d580cec40 tx=0x7f1d5806b6b0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.166+0000 7f1d6d821740 1 -- 172.21.15.132:0/2006120091 shutdown_connections 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.166+0000 7f1d6d821740 1 --2- 172.21.15.132:0/2006120091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6cddd0 0x55943c6d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.166+0000 7f1d6d821740 1 --2- 172.21.15.132:0/2006120091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c3760 0x55943c6cd800 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.166+0000 7f1d6d821740 1 --2- 172.21.15.132:0/2006120091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6c2db0 0x55943c6c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.166+0000 7f1d6d821740 1 -- 172.21.15.132:0/2006120091 >> 172.21.15.132:0/2006120091 conn(0x55943c6bf070 msgr2=0x55943c6bf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.166+0000 7f1d6d821740 1 -- 172.21.15.132:0/2006120091 shutdown_connections 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.166+0000 7f1d6d821740 1 -- 172.21.15.132:0/2006120091 wait complete. 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.166+0000 7f1d6d821740 1 -- start start 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d6d821740 1 Processor -- start 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d6d821740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d6d821740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6c3760 0x55943c6cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d6d821740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6cddd0 0x55943c6cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d6d821740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55943c6d0c80 con 0x55943c6c2db0 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d6d821740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55943c5c3470 con 0x55943c6cddd0 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d6d821740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55943c6d0e00 con 0x55943c6c3760 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d688ab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d6d821740 10 client.0 _ll_register_callbacks cb 0x55943c5e8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:12.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d688ab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58818/0 (socket says 172.21.15.132:58818) 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d688ab640 1 -- 172.21.15.132:0/2455668750 learned_addr learned my addr 172.21.15.132:0/2455668750 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d690ac640 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6cddd0 0x55943c6cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d63fff640 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6c3760 0x55943c6cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d688ab640 1 -- 172.21.15.132:0/2455668750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6c3760 msgr2=0x55943c6cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d688ab640 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6c3760 0x55943c6cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d688ab640 1 -- 172.21.15.132:0/2455668750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6cddd0 msgr2=0x55943c6cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d688ab640 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6cddd0 0x55943c6cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d688ab640 1 -- 172.21.15.132:0/2455668750 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55943c4056b0 con 0x55943c6c2db0 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d690ac640 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6cddd0 0x55943c6cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d688ab640 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6cad40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1d5809c940 tx=0x7f1d5801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f90446492e66b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.167+0000 7f1d43fff640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d43fff640 1 -- 172.21.15.132:0/2455668750 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5806b020 con 0x55943c6c2db0 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d43fff640 1 -- 172.21.15.132:0/2455668750 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f1d58010a90 con 0x55943c6c2db0 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d6d821740 1 -- 172.21.15.132:0/2455668750 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d58087040 con 0x55943c6c2db0 2024-04-23T18:36:12.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d43fff640 1 -- 172.21.15.132:0/2455668750 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d5809d340 con 0x55943c6c2db0 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d6d821740 10 client.15648 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d6d821740 1 -- 172.21.15.132:0/2455668750 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55943c6dcf80 con 0x55943c6c2db0 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d43fff640 1 -- 172.21.15.132:0/2455668750 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d58087040 con 0x55943c6c2db0 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d43fff640 1 -- 172.21.15.132:0/2455668750 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1d580a5420 con 0x55943c6c2db0 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d6d821740 10 client.15648 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d6d821740 10 client.15648 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d6d821740 20 client.15648 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d6d821740 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55943c6df1d0 0x55943c6e0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d6d821740 4 client.15648 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.168+0000 7f1d6d821740 1 -- 172.21.15.132:0/2455668750 --> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55943c636670 con 0x55943c6df1d0 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.169+0000 7f1d63fff640 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55943c6df1d0 0x55943c6e0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.169+0000 7f1d43fff640 10 client.15648 ms_handle_connect on v2:172.21.15.148:6836/2626530887 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.169+0000 7f1d63fff640 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55943c6df1d0 0x55943c6e0640 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3343a0fceceeb804 server_cookie=ec41f5dc0be46a12 in_seq=0 out_seq=0 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d43fff640 1 -- 172.21.15.132:0/2455668750 <== mds.1 v2:172.21.15.148:6836/2626530887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55943c636670 con 0x55943c6df1d0 2024-04-23T18:36:12.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d43fff640 10 client.15648 handle_command_reply: tid=0 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d6d821740 1 client.15648 shutdown 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d6d821740 1 -- 172.21.15.132:0/2455668750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 msgr2=0x55943c6cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d6d821740 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6cad40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1d5809c940 tx=0x7f1d5801f000 comp rx=0 tx=0).stop 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d6d821740 1 -- 172.21.15.132:0/2455668750 shutdown_connections 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d6d821740 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55943c6df1d0 0x55943c6e0640 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d6d821740 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6c3760 0x55943c6cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d6d821740 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6cddd0 0x55943c6cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d43fff640 0 client.15648 ms_handle_reset on v2:172.21.15.148:6836/2626530887 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d6d821740 1 --2- 172.21.15.132:0/2455668750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6cad40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d6d821740 1 -- 172.21.15.132:0/2455668750 >> 172.21.15.132:0/2455668750 conn(0x55943c6bf070 msgr2=0x55943c6c4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d6d821740 1 -- 172.21.15.132:0/2455668750 shutdown_connections 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d6d821740 1 -- 172.21.15.132:0/2455668750 wait complete. 2024-04-23T18:36:12.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.170+0000 7f1d6d821740 20 client.15648 trim_cache size 0 max 16384 2024-04-23T18:36:12.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.187+0000 7f1d6d821740 1 Processor -- start 2024-04-23T18:36:12.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d6d821740 1 -- start start 2024-04-23T18:36:12.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d6d821740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6be950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d6d821740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6c3760 0x55943c6cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d6d821740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6cddd0 0x55943c6c5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d6d821740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55943c6de980 con 0x55943c6c2db0 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d6d821740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55943c6df050 con 0x55943c6c3760 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d6d821740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55943c6df720 con 0x55943c6cddd0 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d688ab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6be950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d688ab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6be950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58832/0 (socket says 172.21.15.132:58832) 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d688ab640 1 -- 172.21.15.132:0/1581333861 learned_addr learned my addr 172.21.15.132:0/1581333861 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d688ab640 1 -- 172.21.15.132:0/1581333861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6cddd0 msgr2=0x55943c6c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d688ab640 1 --2- 172.21.15.132:0/1581333861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6cddd0 0x55943c6c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d688ab640 1 -- 172.21.15.132:0/1581333861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6c3760 msgr2=0x55943c6cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d688ab640 1 --2- 172.21.15.132:0/1581333861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6c3760 0x55943c6cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d688ab640 1 -- 172.21.15.132:0/1581333861 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55943c6bee90 con 0x55943c6c2db0 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d688ab640 1 --2- 172.21.15.132:0/1581333861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6be950 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1d580990a0 tx=0x7f1d58018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a6226ea3a82fce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d4e7fc640 1 -- 172.21.15.132:0/1581333861 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d580ce400 con 0x55943c6c2db0 2024-04-23T18:36:12.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d4e7fc640 1 -- 172.21.15.132:0/1581333861 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d58087040 con 0x55943c6c2db0 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.188+0000 7f1d4e7fc640 1 -- 172.21.15.132:0/1581333861 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58010a90 con 0x55943c6c2db0 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.189+0000 7f1d6d821740 1 -- 172.21.15.132:0/1581333861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 msgr2=0x55943c6be950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.189+0000 7f1d6d821740 1 --2- 172.21.15.132:0/1581333861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6be950 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1d580990a0 tx=0x7f1d58018420 comp rx=0 tx=0).stop 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.189+0000 7f1d6d821740 1 -- 172.21.15.132:0/1581333861 shutdown_connections 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.189+0000 7f1d6d821740 1 --2- 172.21.15.132:0/1581333861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6cddd0 0x55943c6c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.189+0000 7f1d6d821740 1 --2- 172.21.15.132:0/1581333861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6c3760 0x55943c6cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.189+0000 7f1d6d821740 1 --2- 172.21.15.132:0/1581333861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6be950 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.189+0000 7f1d6d821740 1 -- 172.21.15.132:0/1581333861 >> 172.21.15.132:0/1581333861 conn(0x55943c6bf070 msgr2=0x55943c6c9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.189+0000 7f1d6d821740 1 -- 172.21.15.132:0/1581333861 shutdown_connections 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.189+0000 7f1d6d821740 1 -- 172.21.15.132:0/1581333861 wait complete. 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.189+0000 7f1d6d821740 1 -- start start 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.189+0000 7f1d6d821740 1 Processor -- start 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d6d821740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d6d821740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6c3760 0x55943c6c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d6d821740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6cddd0 0x55943c6c8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d6d821740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55943c6e04c0 con 0x55943c6c2db0 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d6d821740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55943c6dfdf0 con 0x55943c6cddd0 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d6d821740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55943c6e0b90 con 0x55943c6c3760 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d6d821740 10 client.0 _ll_register_callbacks cb 0x7f1d50005ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d688ab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d688ab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58834/0 (socket says 172.21.15.132:58834) 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d688ab640 1 -- 172.21.15.132:0/4257961154 learned_addr learned my addr 172.21.15.132:0/4257961154 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d63fff640 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6c3760 0x55943c6c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d688ab640 1 -- 172.21.15.132:0/4257961154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6c3760 msgr2=0x55943c6c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d690ac640 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6cddd0 0x55943c6c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d688ab640 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6c3760 0x55943c6c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d688ab640 1 -- 172.21.15.132:0/4257961154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6cddd0 msgr2=0x55943c6c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d688ab640 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6cddd0 0x55943c6c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d688ab640 1 -- 172.21.15.132:0/4257961154 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55943c6e6ca0 con 0x55943c6c2db0 2024-04-23T18:36:12.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d63fff640 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6c3760 0x55943c6c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d690ac640 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6cddd0 0x55943c6c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d688ab640 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6c7b90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800cf60 tx=0x7f1d5809b150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a949a43b9e56a689 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d43fff640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d43fff640 1 -- 172.21.15.132:0/4257961154 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d580a0ff0 con 0x55943c6c2db0 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d43fff640 1 -- 172.21.15.132:0/4257961154 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f1d580bd0d0 con 0x55943c6c2db0 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d6d821740 1 -- 172.21.15.132:0/4257961154 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d58087040 con 0x55943c6c2db0 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d43fff640 1 -- 172.21.15.132:0/4257961154 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d580a58b0 con 0x55943c6c2db0 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d6d821740 10 client.15654 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d6d821740 1 -- 172.21.15.132:0/4257961154 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55943c6e6ca0 con 0x55943c6c2db0 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.190+0000 7f1d43fff640 1 -- 172.21.15.132:0/4257961154 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d58087040 con 0x55943c6c2db0 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.191+0000 7f1d43fff640 1 -- 172.21.15.132:0/4257961154 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1d58098b50 con 0x55943c6c2db0 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.191+0000 7f1d6d821740 10 client.15654 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.191+0000 7f1d6d821740 10 client.15654 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:36:12.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.191+0000 7f1d6d821740 20 client.15654 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.191+0000 7f1d6d821740 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55943c6df1d0 0x55943c7485a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.191+0000 7f1d6d821740 4 client.15654 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.191+0000 7f1d6d821740 1 -- 172.21.15.132:0/4257961154 --> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] -- command(tid 0: {"prefix": "status"}) -- 0x55943c6304e0 con 0x55943c6df1d0 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.191+0000 7f1d63fff640 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55943c6df1d0 0x55943c7485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.191+0000 7f1d63fff640 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55943c6df1d0 0x55943c7485a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60cc67f39449526c server_cookie=9bd3ebe1fcf940b6 in_seq=0 out_seq=0 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.191+0000 7f1d43fff640 10 client.15654 ms_handle_connect on v2:172.21.15.148:6836/2626530887 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d43fff640 1 -- 172.21.15.132:0/4257961154 <== mds.1 v2:172.21.15.148:6836/2626530887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55943c6304e0 con 0x55943c6df1d0 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d43fff640 10 client.15654 handle_command_reply: tid=0 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6d821740 1 client.15654 shutdown 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6d821740 1 -- 172.21.15.132:0/4257961154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 msgr2=0x55943c6c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6d821740 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6c7b90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800cf60 tx=0x7f1d5809b150 comp rx=0 tx=0).stop 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6d821740 1 -- 172.21.15.132:0/4257961154 shutdown_connections 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6d821740 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.148:6836/2626530887,v1:172.21.15.148:6837/2626530887] conn(0x55943c6df1d0 0x55943c7485a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6d821740 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55943c6c3760 0x55943c6c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6d821740 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55943c6cddd0 0x55943c6c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d43fff640 0 client.15654 ms_handle_reset on v2:172.21.15.148:6836/2626530887 2024-04-23T18:36:12.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6d821740 1 --2- 172.21.15.132:0/4257961154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55943c6c2db0 0x55943c6c7b90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6d821740 1 -- 172.21.15.132:0/4257961154 >> 172.21.15.132:0/4257961154 conn(0x55943c6bf070 msgr2=0x55943c6cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6d821740 1 -- 172.21.15.132:0/4257961154 shutdown_connections 2024-04-23T18:36:12.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6d821740 1 -- 172.21.15.132:0/4257961154 wait complete. 2024-04-23T18:36:12.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6d821740 20 client.15654 trim_cache size 0 max 16384 2024-04-23T18:36:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:36:12.191 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:36:12.191 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 1, 2024-04-23T18:36:12.191 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24491, 2024-04-23T18:36:12.192 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:36:12.192 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:36:12.192 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:36:12.192 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 128.66406491800001, 2024-04-23T18:36:12.192 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 26, 2024-04-23T18:36:12.192 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 91, 2024-04-23T18:36:12.192 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 90, 2024-04-23T18:36:12.192 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 149.151720388 2024-04-23T18:36:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:36:12.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/636668757 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1d540a3e40 msgr2=0x7f1d540a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.192+0000 7f1d6ab0f640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1d540a3e40 0x7f1d540a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c02f540 tx=0x7f1d5c032d00 comp rx=0 tx=0).stop 2024-04-23T18:36:12.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.193+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/636668757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64101130 msgr2=0x7f1d641986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.193+0000 7f1d6ab0f640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64101130 0x7f1d641986e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800e9f0 tx=0x7f1d5800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.193+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/636668757 shutdown_connections 2024-04-23T18:36:12.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.193+0000 7f1d6ab0f640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1d540a3e40 0x7f1d540a6300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.193+0000 7f1d6ab0f640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d64106200 0x7f1d64198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.193+0000 7f1d6ab0f640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d6419cfa0 0x7f1d6419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.193+0000 7f1d6ab0f640 1 --2- 172.21.15.132:0/636668757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64101130 0x7f1d641986e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.193+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/636668757 >> 172.21.15.132:0/636668757 conn(0x7f1d64077f80 msgr2=0x7f1d64104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.193+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/636668757 shutdown_connections 2024-04-23T18:36:12.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.193+0000 7f1d6ab0f640 1 -- 172.21.15.132:0/636668757 wait complete. 2024-04-23T18:36:12.195 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T18:36:12.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.303+0000 7f8704343640 1 -- 172.21.15.132:0/1569106335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86fc101a30 msgr2=0x7f86fc105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.303+0000 7f8704343640 1 --2- 172.21.15.132:0/1569106335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86fc101a30 0x7f86fc105a70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009900 tx=0x7f86ec02a410 comp rx=0 tx=0).stop 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.303+0000 7f8704343640 1 -- 172.21.15.132:0/1569106335 shutdown_connections 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.303+0000 7f8704343640 1 --2- 172.21.15.132:0/1569106335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86fc106150 0x7f86fc10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.303+0000 7f8704343640 1 --2- 172.21.15.132:0/1569106335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86fc101a30 0x7f86fc105a70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.303+0000 7f8704343640 1 --2- 172.21.15.132:0/1569106335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86fc101080 0x7f86fc101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.303+0000 7f8704343640 1 -- 172.21.15.132:0/1569106335 >> 172.21.15.132:0/1569106335 conn(0x7f86fc0771d0 msgr2=0x7f86fc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.303+0000 7f8704343640 1 -- 172.21.15.132:0/1569106335 shutdown_connections 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.303+0000 7f8704343640 1 -- 172.21.15.132:0/1569106335 wait complete. 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f8704343640 1 Processor -- start 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f8704343640 1 -- start start 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f8704343640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86fc101080 0x7f86fc19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f8704343640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86fc106150 0x7f86fc19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f8704343640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86fc1985e0 0x7f86fc198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f8704343640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc111090 con 0x7f86fc1985e0 2024-04-23T18:36:12.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f8704343640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc110f10 con 0x7f86fc106150 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f8704343640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc111210 con 0x7f86fc101080 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f87028e0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86fc1985e0 0x7f86fc198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f87028e0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86fc1985e0 0x7f86fc198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58860/0 (socket says 172.21.15.132:58860) 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f87028e0640 1 -- 172.21.15.132:0/509959154 learned_addr learned my addr 172.21.15.132:0/509959154 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f87020df640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86fc101080 0x7f86fc19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f87018de640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86fc106150 0x7f86fc19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f87028e0640 1 -- 172.21.15.132:0/509959154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86fc101080 msgr2=0x7f86fc19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f87028e0640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86fc101080 0x7f86fc19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f87028e0640 1 -- 172.21.15.132:0/509959154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86fc106150 msgr2=0x7f86fc19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.304+0000 7f87028e0640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86fc106150 0x7f86fc19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.305+0000 7f87028e0640 1 -- 172.21.15.132:0/509959154 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc199100 con 0x7f86fc1985e0 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.305+0000 7f87028e0640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86fc1985e0 0x7f86fc198a90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f86f400eb10 tx=0x7f86f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.305+0000 7f86f37fe640 1 -- 172.21.15.132:0/509959154 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4004b20 con 0x7f86fc1985e0 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.305+0000 7f8704343640 1 -- 172.21.15.132:0/509959154 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc1993f0 con 0x7f86fc1985e0 2024-04-23T18:36:12.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.305+0000 7f86f37fe640 1 -- 172.21.15.132:0/509959154 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f4004cc0 con 0x7f86fc1985e0 2024-04-23T18:36:12.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.305+0000 7f86f37fe640 1 -- 172.21.15.132:0/509959154 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4010670 con 0x7f86fc1985e0 2024-04-23T18:36:12.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.305+0000 7f8704343640 1 -- 172.21.15.132:0/509959154 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc1a55a0 con 0x7f86fc1985e0 2024-04-23T18:36:12.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.307+0000 7f86f37fe640 1 -- 172.21.15.132:0/509959154 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f86f4005020 con 0x7f86fc1985e0 2024-04-23T18:36:12.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.307+0000 7f8707056740 1 Processor -- start 2024-04-23T18:36:12.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f8707056740 1 -- start start 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f8707056740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c03adb0 0x56149c03b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f8707056740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03b760 0x56149c045800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f8707056740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c045dd0 0x56149c048220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f8707056740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56149bfa0c50 con 0x56149c045dd0 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f8707056740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56149bf3d830 con 0x56149c03adb0 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f8707056740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56149bf9cc10 con 0x56149c03b760 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f86f37fe640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f86e80a3e40 0x7f86e80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87018de640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c045dd0 0x56149c048220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87018de640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c045dd0 0x56149c048220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58872/0 (socket says 172.21.15.132:58872) 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87018de640 1 -- 172.21.15.132:0/3908760270 learned_addr learned my addr 172.21.15.132:0/3908760270 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87028e0640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f86e80a3e40 0x7f86e80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87018de640 1 --2- 172.21.15.132:0/3908760270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c03adb0 0x56149c03b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f86f37fe640 1 -- 172.21.15.132:0/509959154 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f86f40d1070 con 0x7f86fc1985e0 2024-04-23T18:36:12.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87020df640 1 --2- 172.21.15.132:0/3908760270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03b760 0x56149c045800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87018de640 1 -- 172.21.15.132:0/3908760270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03b760 msgr2=0x56149c045800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87018de640 1 --2- 172.21.15.132:0/3908760270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03b760 0x56149c045800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87018de640 1 -- 172.21.15.132:0/3908760270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c03adb0 msgr2=0x56149c03b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87018de640 1 --2- 172.21.15.132:0/3908760270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c03adb0 0x56149c03b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87018de640 1 -- 172.21.15.132:0/3908760270 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56149bcb32d0 con 0x56149c045dd0 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87028e0640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f86e80a3e40 0x7f86e80a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86f40ce690 tx=0x7f86f406b760 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.308+0000 7f87018de640 1 --2- 172.21.15.132:0/3908760270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c045dd0 0x56149c048220 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f86ec005c40 tx=0x7f86ec0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2b5f6acc6129b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.309+0000 7f86f0bfe640 1 -- 172.21.15.132:0/3908760270 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec035040 con 0x56149c045dd0 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.309+0000 7f86f0bfe640 1 -- 172.21.15.132:0/3908760270 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec0318c0 con 0x56149c045dd0 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.309+0000 7f86f0bfe640 1 -- 172.21.15.132:0/3908760270 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec003ca0 con 0x56149c045dd0 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.309+0000 7f8707056740 1 -- 172.21.15.132:0/3908760270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c045dd0 msgr2=0x56149c048220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.309+0000 7f8707056740 1 --2- 172.21.15.132:0/3908760270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c045dd0 0x56149c048220 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f86ec005c40 tx=0x7f86ec0058d0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.309+0000 7f8707056740 1 -- 172.21.15.132:0/3908760270 shutdown_connections 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.309+0000 7f8707056740 1 --2- 172.21.15.132:0/3908760270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03b760 0x56149c045800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.309+0000 7f8707056740 1 --2- 172.21.15.132:0/3908760270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c045dd0 0x56149c048220 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.309+0000 7f8707056740 1 --2- 172.21.15.132:0/3908760270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c03adb0 0x56149c03b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.309+0000 7f8707056740 1 -- 172.21.15.132:0/3908760270 >> 172.21.15.132:0/3908760270 conn(0x56149c037070 msgr2=0x56149c037450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f8707056740 1 -- 172.21.15.132:0/3908760270 shutdown_connections 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f8707056740 1 -- 172.21.15.132:0/3908760270 wait complete. 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f8707056740 1 -- start start 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f8707056740 1 Processor -- start 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f8707056740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03adb0 0x56149c042d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f8707056740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03b760 0x56149c043270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f8707056740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c045dd0 0x56149c0437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f8707056740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56149c048ce0 con 0x56149c03adb0 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f8707056740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56149bf3b470 con 0x56149c045dd0 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f8707056740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56149c048e60 con 0x56149c03b760 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f8707056740 10 client.0 _ll_register_callbacks cb 0x56149bf60f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f87018de640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03adb0 0x56149c042d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f87018de640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03adb0 0x56149c042d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58888/0 (socket says 172.21.15.132:58888) 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f87018de640 1 -- 172.21.15.132:0/775537558 learned_addr learned my addr 172.21.15.132:0/775537558 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f87020df640 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03b760 0x56149c043270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f87018de640 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c045dd0 0x56149c0437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f87018de640 1 -- 172.21.15.132:0/775537558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03b760 msgr2=0x56149c043270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f87018de640 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03b760 0x56149c043270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f87018de640 1 -- 172.21.15.132:0/775537558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c045dd0 msgr2=0x56149c0437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f87018de640 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c045dd0 0x56149c0437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.310+0000 7f87018de640 1 -- 172.21.15.132:0/775537558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56149bd7d6b0 con 0x56149c03adb0 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f87018de640 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03adb0 0x56149c042d30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f86ec005ec0 tx=0x7f86ec032e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1088b9733e6c0880 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f86e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f86e5ffb640 1 -- 172.21.15.132:0/775537558 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec035070 con 0x56149c03adb0 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f86e5ffb640 1 -- 172.21.15.132:0/775537558 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f86ec03d690 con 0x56149c03adb0 2024-04-23T18:36:12.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f8707056740 1 -- 172.21.15.132:0/775537558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ec0318c0 con 0x56149c03adb0 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f86e5ffb640 1 -- 172.21.15.132:0/775537558 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f86ec03d890 con 0x56149c03adb0 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f8707056740 10 client.15666 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f8707056740 1 -- 172.21.15.132:0/775537558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56149c055010 con 0x56149c03adb0 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f86e5ffb640 1 -- 172.21.15.132:0/775537558 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec0318c0 con 0x56149c03adb0 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f86e5ffb640 1 -- 172.21.15.132:0/775537558 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f86ec03a040 con 0x56149c03adb0 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f8707056740 10 client.15666 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f8707056740 10 client.15666 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f8707056740 20 client.15666 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f8707056740 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56149c056360 0x56149c058560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f8707056740 4 client.15666 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.311+0000 7f8707056740 1 -- 172.21.15.132:0/775537558 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56149bfae670 con 0x56149c056360 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.312+0000 7f87020df640 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56149c056360 0x56149c058560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.312+0000 7f86e5ffb640 10 client.15666 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.312+0000 7f87020df640 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56149c056360 0x56149c058560 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a883801403dd36 server_cookie=fdf8065a54f5fc4e in_seq=0 out_seq=0 2024-04-23T18:36:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f86e5ffb640 1 -- 172.21.15.132:0/775537558 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56149bfae670 con 0x56149c056360 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f86e5ffb640 10 client.15666 handle_command_reply: tid=0 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f8707056740 1 client.15666 shutdown 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f8707056740 1 -- 172.21.15.132:0/775537558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03adb0 msgr2=0x56149c042d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f8707056740 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03adb0 0x56149c042d30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f86ec005ec0 tx=0x7f86ec032e00 comp rx=0 tx=0).stop 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f8707056740 1 -- 172.21.15.132:0/775537558 shutdown_connections 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f8707056740 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56149c056360 0x56149c058560 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f8707056740 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03b760 0x56149c043270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f86e5ffb640 0 client.15666 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f8707056740 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c045dd0 0x56149c0437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f8707056740 1 --2- 172.21.15.132:0/775537558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03adb0 0x56149c042d30 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f8707056740 1 -- 172.21.15.132:0/775537558 >> 172.21.15.132:0/775537558 conn(0x56149c037070 msgr2=0x56149c03cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f8707056740 1 -- 172.21.15.132:0/775537558 shutdown_connections 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f8707056740 1 -- 172.21.15.132:0/775537558 wait complete. 2024-04-23T18:36:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.313+0000 7f8707056740 20 client.15666 trim_cache size 0 max 16384 2024-04-23T18:36:12.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f8707056740 1 Processor -- start 2024-04-23T18:36:12.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f8707056740 1 -- start start 2024-04-23T18:36:12.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f8707056740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c03adb0 0x56149c03b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f8707056740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 0x56149c03ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f8707056740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c045dd0 0x56149c0404d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f8707056740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56149c0568a0 con 0x56149c03b760 2024-04-23T18:36:12.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f8707056740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56149c056f70 con 0x56149c03adb0 2024-04-23T18:36:12.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f8707056740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56149c057640 con 0x56149c045dd0 2024-04-23T18:36:12.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f87020df640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 0x56149c03ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f87020df640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 0x56149c03ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58892/0 (socket says 172.21.15.132:58892) 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f87020df640 1 -- 172.21.15.132:0/189916009 learned_addr learned my addr 172.21.15.132:0/189916009 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f87018de640 1 --2- 172.21.15.132:0/189916009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c045dd0 0x56149c0404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f87018de640 1 --2- 172.21.15.132:0/189916009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c03adb0 0x56149c03b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f87020df640 1 -- 172.21.15.132:0/189916009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c045dd0 msgr2=0x56149c0404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f87020df640 1 --2- 172.21.15.132:0/189916009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c045dd0 0x56149c0404d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f87020df640 1 -- 172.21.15.132:0/189916009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c03adb0 msgr2=0x56149c03b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f87020df640 1 --2- 172.21.15.132:0/189916009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c03adb0 0x56149c03b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f87020df640 1 -- 172.21.15.132:0/189916009 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56149c041280 con 0x56149c03b760 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f87018de640 1 --2- 172.21.15.132:0/189916009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c045dd0 0x56149c0404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.331+0000 7f87020df640 1 --2- 172.21.15.132:0/189916009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 0x56149c03ff90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009d70 tx=0x7f86f8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54c4824d0fa3ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f86e7fff640 1 -- 172.21.15.132:0/189916009 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8004310 con 0x56149c03b760 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f86e7fff640 1 -- 172.21.15.132:0/189916009 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f80044b0 con 0x56149c03b760 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f86e7fff640 1 -- 172.21.15.132:0/189916009 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8004e50 con 0x56149c03b760 2024-04-23T18:36:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f8707056740 1 -- 172.21.15.132:0/189916009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 msgr2=0x56149c03ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f8707056740 1 --2- 172.21.15.132:0/189916009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 0x56149c03ff90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009d70 tx=0x7f86f8005f90 comp rx=0 tx=0).stop 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f8707056740 1 -- 172.21.15.132:0/189916009 shutdown_connections 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f8707056740 1 --2- 172.21.15.132:0/189916009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c045dd0 0x56149c0404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f8707056740 1 --2- 172.21.15.132:0/189916009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 0x56149c03ff90 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f8707056740 1 --2- 172.21.15.132:0/189916009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c03adb0 0x56149c03b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f8707056740 1 -- 172.21.15.132:0/189916009 >> 172.21.15.132:0/189916009 conn(0x56149c037070 msgr2=0x56149c041600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f8707056740 1 -- 172.21.15.132:0/189916009 shutdown_connections 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f8707056740 1 -- 172.21.15.132:0/189916009 wait complete. 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f8707056740 1 -- start start 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.332+0000 7f8707056740 1 Processor -- start 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f8707056740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03adb0 0x56149c05a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f8707056740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 0x56149c05a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f8707056740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c045dd0 0x56149c05ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f8707056740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56149c0583e0 con 0x56149c03b760 2024-04-23T18:36:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f8707056740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56149c057d10 con 0x56149c045dd0 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f8707056740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56149c058ab0 con 0x56149c03adb0 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f8707056740 10 client.0 _ll_register_callbacks cb 0x7f86f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f87020df640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 0x56149c05a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f87020df640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 0x56149c05a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58906/0 (socket says 172.21.15.132:58906) 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f87020df640 1 -- 172.21.15.132:0/3563167554 learned_addr learned my addr 172.21.15.132:0/3563167554 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f87018de640 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c045dd0 0x56149c05ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f87018de640 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03adb0 0x56149c05a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f87020df640 1 -- 172.21.15.132:0/3563167554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03adb0 msgr2=0x56149c05a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f87020df640 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03adb0 0x56149c05a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f87020df640 1 -- 172.21.15.132:0/3563167554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c045dd0 msgr2=0x56149c05ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f87020df640 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c045dd0 0x56149c05ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f87020df640 1 -- 172.21.15.132:0/3563167554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56149c05eb70 con 0x56149c03b760 2024-04-23T18:36:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f87018de640 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c045dd0 0x56149c05ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f87020df640 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 0x56149c05a840 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009710 tx=0x7f86f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2f6683e72b99e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f86e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f86e5ffb640 1 -- 172.21.15.132:0/3563167554 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8004e60 con 0x56149c03b760 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f86e5ffb640 1 -- 172.21.15.132:0/3563167554 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f86f80048c0 con 0x56149c03b760 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.333+0000 7f8707056740 1 -- 172.21.15.132:0/3563167554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f80044b0 con 0x56149c03b760 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.334+0000 7f86e5ffb640 1 -- 172.21.15.132:0/3563167554 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f86f8037050 con 0x56149c03b760 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.334+0000 7f8707056740 10 client.15672 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.334+0000 7f8707056740 1 -- 172.21.15.132:0/3563167554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56149c0bd470 con 0x56149c03b760 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.334+0000 7f86e5ffb640 1 -- 172.21.15.132:0/3563167554 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f80044b0 con 0x56149c03b760 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.334+0000 7f86e5ffb640 1 -- 172.21.15.132:0/3563167554 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f86f8003a90 con 0x56149c03b760 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.334+0000 7f8707056740 10 client.15672 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.334+0000 7f8707056740 10 client.15672 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.334+0000 7f8707056740 20 client.15672 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.334+0000 7f8707056740 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56149c0c1da0 0x56149c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.334+0000 7f8707056740 4 client.15672 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.334+0000 7f8707056740 1 -- 172.21.15.132:0/3563167554 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "status"}) -- 0x56149bfa84e0 con 0x56149c0c1da0 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.334+0000 7f87018de640 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56149c0c1da0 0x56149c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f87018de640 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56149c0c1da0 0x56149c0c0ab0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e2a87937bae18eb server_cookie=4304530e95c1777 in_seq=0 out_seq=0 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f86e5ffb640 10 client.15672 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f86e5ffb640 1 -- 172.21.15.132:0/3563167554 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56149bfa84e0 con 0x56149c0c1da0 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f86e5ffb640 10 client.15672 handle_command_reply: tid=0 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8707056740 1 client.15672 shutdown 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8707056740 1 -- 172.21.15.132:0/3563167554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 msgr2=0x56149c05a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8707056740 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 0x56149c05a840 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009710 tx=0x7f86f80042f0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8707056740 1 -- 172.21.15.132:0/3563167554 shutdown_connections 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8707056740 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56149c0c1da0 0x56149c0c0ab0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8707056740 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56149c045dd0 0x56149c05ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8707056740 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149c03b760 0x56149c05a840 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f86e5ffb640 0 client.15672 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:12.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8707056740 1 --2- 172.21.15.132:0/3563167554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56149c03adb0 0x56149c05a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8707056740 1 -- 172.21.15.132:0/3563167554 >> 172.21.15.132:0/3563167554 conn(0x56149c037070 msgr2=0x56149c042cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8707056740 1 -- 172.21.15.132:0/3563167554 shutdown_connections 2024-04-23T18:36:12.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8707056740 1 -- 172.21.15.132:0/3563167554 wait complete. 2024-04-23T18:36:12.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8707056740 20 client.15672 trim_cache size 0 max 16384 2024-04-23T18:36:12.337 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:36:12.337 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "c34642b4-019e-11ef-bc93-c7b262605968", 2024-04-23T18:36:12.337 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 0, 2024-04-23T18:36:12.337 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24515, 2024-04-23T18:36:12.338 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-04-23T18:36:12.338 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-04-23T18:36:12.338 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-04-23T18:36:12.338 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 130.81933851599999, 2024-04-23T18:36:12.338 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 26, 2024-04-23T18:36:12.338 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 91, 2024-04-23T18:36:12.338 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 90, 2024-04-23T18:36:12.338 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 137.81450595000001 2024-04-23T18:36:12.338 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:36:12.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8704343640 1 -- 172.21.15.132:0/509959154 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f86e80a3e40 msgr2=0x7f86e80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8704343640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f86e80a3e40 0x7f86e80a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86f40ce690 tx=0x7f86f406b760 comp rx=0 tx=0).stop 2024-04-23T18:36:12.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8704343640 1 -- 172.21.15.132:0/509959154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86fc1985e0 msgr2=0x7f86fc198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:12.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.335+0000 7f8704343640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86fc1985e0 0x7f86fc198a90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f86f400eb10 tx=0x7f86f400c2d0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.336+0000 7f8704343640 1 -- 172.21.15.132:0/509959154 shutdown_connections 2024-04-23T18:36:12.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.336+0000 7f8704343640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f86e80a3e40 0x7f86e80a6300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.336+0000 7f8704343640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86fc1985e0 0x7f86fc198a90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.336+0000 7f8704343640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86fc106150 0x7f86fc19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.336+0000 7f8704343640 1 --2- 172.21.15.132:0/509959154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86fc101080 0x7f86fc19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:12.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.336+0000 7f8704343640 1 -- 172.21.15.132:0/509959154 >> 172.21.15.132:0/509959154 conn(0x7f86fc0771d0 msgr2=0x7f86fc104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:12.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.336+0000 7f8704343640 1 -- 172.21.15.132:0/509959154 shutdown_connections 2024-04-23T18:36:12.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:12.336+0000 7f8704343640 1 -- 172.21.15.132:0/509959154 wait complete. 2024-04-23T18:36:12.341 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-23T18:36:12.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:12 smithi132 ceph-mon[29178]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-04-23T18:36:12.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:12 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1308127461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:13.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:12 smithi148 ceph-mon[32311]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-04-23T18:36:13.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:12 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1308127461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:13.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:12 smithi157 ceph-mon[32580]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-04-23T18:36:13.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:12 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1308127461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:14.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:14 smithi132 ceph-mon[29178]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-04-23T18:36:15.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:14 smithi148 ceph-mon[32311]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-04-23T18:36:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:14 smithi157 ceph-mon[32580]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-04-23T18:36:16.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:16 smithi132 ceph-mon[29178]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-04-23T18:36:17.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:16 smithi148 ceph-mon[32311]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-04-23T18:36:17.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:16 smithi157 ceph-mon[32580]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-04-23T18:36:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.527+0000 7feb28e05640 1 -- 172.21.15.132:0/3932292108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb24106170 msgr2=0x7feb2410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.527+0000 7feb28e05640 1 --2- 172.21.15.132:0/3932292108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb24106170 0x7feb2410dd40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009ca0 tx=0x7feb1c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.527+0000 7feb28e05640 1 -- 172.21.15.132:0/3932292108 shutdown_connections 2024-04-23T18:36:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.527+0000 7feb28e05640 1 --2- 172.21.15.132:0/3932292108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb24101aa0 0x7feb24105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.527+0000 7feb28e05640 1 --2- 172.21.15.132:0/3932292108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb24106170 0x7feb2410dd40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.527+0000 7feb28e05640 1 --2- 172.21.15.132:0/3932292108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb241010f0 0x7feb241014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.527+0000 7feb28e05640 1 -- 172.21.15.132:0/3932292108 >> 172.21.15.132:0/3932292108 conn(0x7feb24077fd0 msgr2=0x7feb240783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.527+0000 7feb28e05640 1 -- 172.21.15.132:0/3932292108 shutdown_connections 2024-04-23T18:36:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.527+0000 7feb28e05640 1 -- 172.21.15.132:0/3932292108 wait complete. 2024-04-23T18:36:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.528+0000 7feb28e05640 1 Processor -- start 2024-04-23T18:36:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.528+0000 7feb28e05640 1 -- start start 2024-04-23T18:36:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.528+0000 7feb28e05640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb241010f0 0x7feb24198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.528+0000 7feb28e05640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb24106170 0x7feb24198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.528+0000 7feb28e05640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb2419cee0 0x7feb2419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.528+0000 7feb28e05640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7feb241110c0 con 0x7feb2419cee0 2024-04-23T18:36:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.528+0000 7feb28e05640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb24110f40 con 0x7feb24106170 2024-04-23T18:36:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.528+0000 7feb28e05640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb24111240 con 0x7feb241010f0 2024-04-23T18:36:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.528+0000 7feb22d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb2419cee0 0x7feb2419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.528+0000 7feb22d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb2419cee0 0x7feb2419d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58926/0 (socket says 172.21.15.132:58926) 2024-04-23T18:36:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.528+0000 7feb22d9d640 1 -- 172.21.15.132:0/4120307838 learned_addr learned my addr 172.21.15.132:0/4120307838 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.528+0000 7feb2259c640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb241010f0 0x7feb24198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.529+0000 7feb1bfff640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb24106170 0x7feb24198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.529+0000 7feb22d9d640 1 -- 172.21.15.132:0/4120307838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb241010f0 msgr2=0x7feb24198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.529+0000 7feb22d9d640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb241010f0 0x7feb24198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.529+0000 7feb22d9d640 1 -- 172.21.15.132:0/4120307838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb24106170 msgr2=0x7feb24198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.529+0000 7feb22d9d640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb24106170 0x7feb24198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.529+0000 7feb22d9d640 1 -- 172.21.15.132:0/4120307838 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2419d9d0 con 0x7feb2419cee0 2024-04-23T18:36:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.529+0000 7feb22d9d640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb2419cee0 0x7feb2419d390 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7feb1c005cc0 tx=0x7feb1c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.529+0000 7feb1b7fe640 1 -- 172.21.15.132:0/4120307838 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c003bf0 con 0x7feb2419cee0 2024-04-23T18:36:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.529+0000 7feb28e05640 1 -- 172.21.15.132:0/4120307838 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2419dc60 con 0x7feb2419cee0 2024-04-23T18:36:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.529+0000 7feb1b7fe640 1 -- 172.21.15.132:0/4120307838 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb1c003d90 con 0x7feb2419cee0 2024-04-23T18:36:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.529+0000 7feb1b7fe640 1 -- 172.21.15.132:0/4120307838 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c02fb20 con 0x7feb2419cee0 2024-04-23T18:36:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.529+0000 7feb28e05640 1 -- 172.21.15.132:0/4120307838 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb241a5480 con 0x7feb2419cee0 2024-04-23T18:36:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.531+0000 7feb2bbf8740 1 Processor -- start 2024-04-23T18:36:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.531+0000 7feb1b7fe640 1 -- 172.21.15.132:0/4120307838 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7feb1c004580 con 0x7feb2419cee0 2024-04-23T18:36:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2bbf8740 1 -- start start 2024-04-23T18:36:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2bbf8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2bbf8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9c9e40 0x5613ce9d86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2bbf8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9d8c10 0x5613ce9db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2bbf8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5613ce92e970 con 0x5613ce9d8c10 2024-04-23T18:36:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2bbf8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5613ce8cb4a0 con 0x5613ce9c9520 2024-04-23T18:36:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2bbf8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5613ce92a930 con 0x5613ce9c9e40 2024-04-23T18:36:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb1bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9d8c10 0x5613ce9db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb1bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9d8c10 0x5613ce9db000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58934/0 (socket says 172.21.15.132:58934) 2024-04-23T18:36:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9c9e40 0x5613ce9d86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb1bfff640 1 -- 172.21.15.132:0/3463748179 learned_addr learned my addr 172.21.15.132:0/3463748179 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb1bfff640 1 --2- 172.21.15.132:0/3463748179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb1b7fe640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7feb100a3ee0 0x7feb100a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2259c640 1 -- 172.21.15.132:0/3463748179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 msgr2=0x5613ce9c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2259c640 1 --2- 172.21.15.132:0/3463748179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb22d9d640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7feb100a3ee0 0x7feb100a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2259c640 1 -- 172.21.15.132:0/3463748179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9d8c10 msgr2=0x5613ce9db000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2259c640 1 --2- 172.21.15.132:0/3463748179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9d8c10 0x5613ce9db000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.532+0000 7feb2259c640 1 -- 172.21.15.132:0/3463748179 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613ce641210 con 0x5613ce9c9e40 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.533+0000 7feb1b7fe640 1 -- 172.21.15.132:0/4120307838 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7feb1c0f1f30 con 0x7feb2419cee0 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.533+0000 7feb2259c640 1 --2- 172.21.15.132:0/3463748179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9c9e40 0x5613ce9d86d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feb1400abf0 tx=0x7feb1400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb5212c0a55602b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.533+0000 7feb18ff9640 1 -- 172.21.15.132:0/3463748179 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14005220 con 0x5613ce9c9e40 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.533+0000 7feb18ff9640 1 -- 172.21.15.132:0/3463748179 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb140047a0 con 0x5613ce9c9e40 2024-04-23T18:36:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.533+0000 7feb18ff9640 1 -- 172.21.15.132:0/3463748179 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140186a0 con 0x5613ce9c9e40 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.533+0000 7feb22d9d640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7feb100a3ee0 0x7feb100a63a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0ef660 tx=0x7feb1c0de040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 -- 172.21.15.132:0/3463748179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9c9e40 msgr2=0x5613ce9d86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/3463748179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9c9e40 0x5613ce9d86d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feb1400abf0 tx=0x7feb1400f790 comp rx=0 tx=0).stop 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 -- 172.21.15.132:0/3463748179 shutdown_connections 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/3463748179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9c9e40 0x5613ce9d86d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/3463748179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9d8c10 0x5613ce9db000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/3463748179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 -- 172.21.15.132:0/3463748179 >> 172.21.15.132:0/3463748179 conn(0x5613ce9c5b30 msgr2=0x5613ce9c5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 -- 172.21.15.132:0/3463748179 shutdown_connections 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 -- 172.21.15.132:0/3463748179 wait complete. 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 -- start start 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 Processor -- start 2024-04-23T18:36:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9d8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9d8c10 0x5613ce9d5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5613ce9db840 con 0x5613ce9c9e40 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5613ce8c90b0 con 0x5613ce9c9520 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5613ce9db9c0 con 0x5613ce9d8c10 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2bbf8740 10 client.0 _ll_register_callbacks cb 0x5613ce8b6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.534+0000 7feb2259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb2259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9d5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58942/0 (socket says 172.21.15.132:58942) 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb2259c640 1 -- 172.21.15.132:0/1087737775 learned_addr learned my addr 172.21.15.132:0/1087737775 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb1bfff640 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9d8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb2259c640 1 -- 172.21.15.132:0/1087737775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9d8c10 msgr2=0x5613ce9d5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb2259c640 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9d8c10 0x5613ce9d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb2259c640 1 -- 172.21.15.132:0/1087737775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 msgr2=0x5613ce9d8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb2259c640 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9d8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb2259c640 1 -- 172.21.15.132:0/1087737775 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613ce70d620 con 0x5613ce9c9e40 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb1bfff640 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9d8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb2259c640 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9d5740 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feb14010010 tx=0x7feb140051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ca1d946191f3b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb09ffb640 1 -- 172.21.15.132:0/1087737775 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14027070 con 0x5613ce9c9e40 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb09ffb640 1 -- 172.21.15.132:0/1087737775 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7feb14005600 con 0x5613ce9c9e40 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb2bbf8740 1 -- 172.21.15.132:0/1087737775 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb140047a0 con 0x5613ce9c9e40 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb09ffb640 1 -- 172.21.15.132:0/1087737775 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feb14033050 con 0x5613ce9c9e40 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb2bbf8740 10 client.15684 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb2bbf8740 1 -- 172.21.15.132:0/1087737775 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5613ce9e76e0 con 0x5613ce9c9e40 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.535+0000 7feb09ffb640 1 -- 172.21.15.132:0/1087737775 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb140047a0 con 0x5613ce9c9e40 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.536+0000 7feb09ffb640 1 -- 172.21.15.132:0/1087737775 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feb14023430 con 0x5613ce9c9e40 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.536+0000 7feb2bbf8740 10 client.15684 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.536+0000 7feb2bbf8740 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.536+0000 7feb2bbf8740 20 client.15684 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.536+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5613ce9e8b50 0x5613ce9eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.536+0000 7feb2bbf8740 4 client.15684 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.536+0000 7feb2bbf8740 1 -- 172.21.15.132:0/1087737775 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613ce93c390 con 0x5613ce9e8b50 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.536+0000 7feb1bfff640 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5613ce9e8b50 0x5613ce9eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.536+0000 7feb09ffb640 10 client.15684 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.536+0000 7feb1bfff640 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5613ce9e8b50 0x5613ce9eaf40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98d7c4a498016d5f server_cookie=1498c9182311bfb7 in_seq=0 out_seq=0 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb09ffb640 1 -- 172.21.15.132:0/1087737775 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613ce93c390 con 0x5613ce9e8b50 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb09ffb640 10 client.15684 handle_command_reply: tid=0 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb2bbf8740 1 client.15684 shutdown 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb2bbf8740 1 -- 172.21.15.132:0/1087737775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 msgr2=0x5613ce9d5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9d5740 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feb14010010 tx=0x7feb140051d0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb2bbf8740 1 -- 172.21.15.132:0/1087737775 shutdown_connections 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5613ce9e8b50 0x5613ce9eaf40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9d8c10 0x5613ce9d5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9d5740 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/1087737775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9d8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb09ffb640 0 client.15684 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb2bbf8740 1 -- 172.21.15.132:0/1087737775 >> 172.21.15.132:0/1087737775 conn(0x5613ce9c5b30 msgr2=0x5613ce9cb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb2bbf8740 1 -- 172.21.15.132:0/1087737775 shutdown_connections 2024-04-23T18:36:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb2bbf8740 1 -- 172.21.15.132:0/1087737775 wait complete. 2024-04-23T18:36:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.537+0000 7feb2bbf8740 20 client.15684 trim_cache size 0 max 16384 2024-04-23T18:36:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.552+0000 7feb2bbf8740 1 Processor -- start 2024-04-23T18:36:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.552+0000 7feb2bbf8740 1 -- start start 2024-04-23T18:36:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.552+0000 7feb2bbf8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9c9520 0x5613ce9e4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.552+0000 7feb2bbf8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.552+0000 7feb2bbf8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9d8c10 0x5613ce9d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.552+0000 7feb2bbf8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7feb241ab8d0 con 0x5613ce9c9e40 2024-04-23T18:36:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.552+0000 7feb2bbf8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb241ad410 con 0x5613ce9d8c10 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.552+0000 7feb2bbf8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb241abfa0 con 0x5613ce9c9520 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.552+0000 7feb2259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.552+0000 7feb2259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9d0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58956/0 (socket says 172.21.15.132:58956) 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.552+0000 7feb2259c640 1 -- 172.21.15.132:0/830067582 learned_addr learned my addr 172.21.15.132:0/830067582 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.552+0000 7feb1bfff640 1 --2- 172.21.15.132:0/830067582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9c9520 0x5613ce9e4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.553+0000 7feb1bfff640 1 --2- 172.21.15.132:0/830067582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9d8c10 0x5613ce9d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.553+0000 7feb2259c640 1 -- 172.21.15.132:0/830067582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9c9520 msgr2=0x5613ce9e4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.553+0000 7feb2259c640 1 --2- 172.21.15.132:0/830067582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9c9520 0x5613ce9e4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.553+0000 7feb2259c640 1 -- 172.21.15.132:0/830067582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9d8c10 msgr2=0x5613ce9d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.553+0000 7feb2259c640 1 --2- 172.21.15.132:0/830067582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9d8c10 0x5613ce9d7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.553+0000 7feb2259c640 1 -- 172.21.15.132:0/830067582 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613ce9c5960 con 0x5613ce9c9e40 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.553+0000 7feb2259c640 1 --2- 172.21.15.132:0/830067582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9d0d80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7feb14000c00 tx=0x7feb1400a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6570187baf2d822 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.553+0000 7feb0bfff640 1 -- 172.21.15.132:0/830067582 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1403c070 con 0x5613ce9c9e40 2024-04-23T18:36:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.553+0000 7feb0bfff640 1 -- 172.21.15.132:0/830067582 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb14005240 con 0x5613ce9c9e40 2024-04-23T18:36:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.553+0000 7feb0bfff640 1 -- 172.21.15.132:0/830067582 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14027070 con 0x5613ce9c9e40 2024-04-23T18:36:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 -- 172.21.15.132:0/830067582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 msgr2=0x5613ce9d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/830067582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9d0d80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7feb14000c00 tx=0x7feb1400a670 comp rx=0 tx=0).stop 2024-04-23T18:36:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 -- 172.21.15.132:0/830067582 shutdown_connections 2024-04-23T18:36:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/830067582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9d8c10 0x5613ce9d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/830067582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9d0d80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/830067582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9c9520 0x5613ce9e4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 -- 172.21.15.132:0/830067582 >> 172.21.15.132:0/830067582 conn(0x5613ce9c5b30 msgr2=0x5613ce9d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 -- 172.21.15.132:0/830067582 shutdown_connections 2024-04-23T18:36:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 -- 172.21.15.132:0/830067582 wait complete. 2024-04-23T18:36:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 -- start start 2024-04-23T18:36:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 Processor -- start 2024-04-23T18:36:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9e0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9d8c10 0x5613ce9d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7feb241acd40 con 0x5613ce9c9e40 2024-04-23T18:36:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb241ac670 con 0x5613ce9c9520 2024-04-23T18:36:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb241a98d0 con 0x5613ce9d8c10 2024-04-23T18:36:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2bbf8740 10 client.0 _ll_register_callbacks cb 0x7feb1400aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.554+0000 7feb2259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9e0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb2259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9e0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58970/0 (socket says 172.21.15.132:58970) 2024-04-23T18:36:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb2259c640 1 -- 172.21.15.132:0/4214942510 learned_addr learned my addr 172.21.15.132:0/4214942510 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb1bfff640 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb1bfff640 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9d8c10 0x5613ce9d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb2259c640 1 -- 172.21.15.132:0/4214942510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9d8c10 msgr2=0x5613ce9d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb2259c640 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9d8c10 0x5613ce9d19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb2259c640 1 -- 172.21.15.132:0/4214942510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 msgr2=0x5613ce9d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb2259c640 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9d3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb2259c640 1 -- 172.21.15.132:0/4214942510 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613ce9d1ef0 con 0x5613ce9c9e40 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb1bfff640 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb2259c640 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9e0c80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7feb1403a040 tx=0x7feb14004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48ceefdef0c6986 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb09ffb640 1 -- 172.21.15.132:0/4214942510 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1403c030 con 0x5613ce9c9e40 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb09ffb640 1 -- 172.21.15.132:0/4214942510 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7feb14023430 con 0x5613ce9c9e40 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb2bbf8740 1 -- 172.21.15.132:0/4214942510 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb14005240 con 0x5613ce9c9e40 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb09ffb640 1 -- 172.21.15.132:0/4214942510 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feb14027040 con 0x5613ce9c9e40 2024-04-23T18:36:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb2bbf8740 10 client.15690 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb2bbf8740 1 -- 172.21.15.132:0/4214942510 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5613cea4e500 con 0x5613ce9c9e40 2024-04-23T18:36:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.555+0000 7feb09ffb640 1 -- 172.21.15.132:0/4214942510 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb14005240 con 0x5613ce9c9e40 2024-04-23T18:36:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.556+0000 7feb09ffb640 1 -- 172.21.15.132:0/4214942510 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feb1402cbc0 con 0x5613ce9c9e40 2024-04-23T18:36:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.556+0000 7feb2bbf8740 10 client.15690 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.556+0000 7feb2bbf8740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.556+0000 7feb2bbf8740 20 client.15690 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.556+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5613cea4ebe0 0x5613cea51000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.556+0000 7feb2bbf8740 4 client.15690 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:36:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.556+0000 7feb2bbf8740 1 -- 172.21.15.132:0/4214942510 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613ce936200 con 0x5613cea4ebe0 2024-04-23T18:36:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.556+0000 7feb1bfff640 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5613cea4ebe0 0x5613cea51000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.556+0000 7feb09ffb640 10 client.15690 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.556+0000 7feb1bfff640 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5613cea4ebe0 0x5613cea51000 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7538ee62074ce4 server_cookie=dcb290cfabbbbeff in_seq=0 out_seq=0 2024-04-23T18:36:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb09ffb640 1 -- 172.21.15.132:0/4214942510 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613ce936200 con 0x5613cea4ebe0 2024-04-23T18:36:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb09ffb640 10 client.15690 handle_command_reply: tid=0 2024-04-23T18:36:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb2bbf8740 1 client.15690 shutdown 2024-04-23T18:36:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb2bbf8740 1 -- 172.21.15.132:0/4214942510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 msgr2=0x5613ce9e0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9e0c80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7feb1403a040 tx=0x7feb14004800 comp rx=0 tx=0).stop 2024-04-23T18:36:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb2bbf8740 1 -- 172.21.15.132:0/4214942510 shutdown_connections 2024-04-23T18:36:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5613cea4ebe0 0x5613cea51000 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb09ffb640 0 client.15690 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5613ce9d8c10 0x5613ce9d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613ce9c9e40 0x5613ce9e0c80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb2bbf8740 1 --2- 172.21.15.132:0/4214942510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613ce9c9520 0x5613ce9d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb2bbf8740 1 -- 172.21.15.132:0/4214942510 >> 172.21.15.132:0/4214942510 conn(0x5613ce9c5b30 msgr2=0x5613ce9c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb2bbf8740 1 -- 172.21.15.132:0/4214942510 shutdown_connections 2024-04-23T18:36:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb2bbf8740 1 -- 172.21.15.132:0/4214942510 wait complete. 2024-04-23T18:36:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.568+0000 7feb2bbf8740 20 client.15690 trim_cache size 0 max 16384 2024-04-23T18:36:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-04-23T18:36:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-04-23T18:36:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-04-23T18:36:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-04-23T18:36:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.569+0000 7feb28e05640 1 -- 172.21.15.132:0/4120307838 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7feb100a3ee0 msgr2=0x7feb100a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.569+0000 7feb28e05640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7feb100a3ee0 0x7feb100a63a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0ef660 tx=0x7feb1c0de040 comp rx=0 tx=0).stop 2024-04-23T18:36:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.569+0000 7feb28e05640 1 -- 172.21.15.132:0/4120307838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb2419cee0 msgr2=0x7feb2419d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.569+0000 7feb28e05640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb2419cee0 0x7feb2419d390 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7feb1c005cc0 tx=0x7feb1c02f380 comp rx=0 tx=0).stop 2024-04-23T18:36:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.569+0000 7feb28e05640 1 -- 172.21.15.132:0/4120307838 shutdown_connections 2024-04-23T18:36:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.569+0000 7feb28e05640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7feb100a3ee0 0x7feb100a63a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.569+0000 7feb28e05640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb2419cee0 0x7feb2419d390 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.569+0000 7feb28e05640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb24106170 0x7feb24198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.569+0000 7feb28e05640 1 --2- 172.21.15.132:0/4120307838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb241010f0 0x7feb24198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.569+0000 7feb28e05640 1 -- 172.21.15.132:0/4120307838 >> 172.21.15.132:0/4120307838 conn(0x7feb24077fd0 msgr2=0x7feb24104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.569+0000 7feb28e05640 1 -- 172.21.15.132:0/4120307838 shutdown_connections 2024-04-23T18:36:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:18.569+0000 7feb28e05640 1 -- 172.21.15.132:0/4120307838 wait complete. 2024-04-23T18:36:18.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:18 smithi132 ceph-mon[29178]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-23T18:36:19.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:18 smithi148 ceph-mon[32311]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-23T18:36:19.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:18 smithi157 ceph-mon[32580]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-23T18:36:20.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:20 smithi132 ceph-mon[29178]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-04-23T18:36:21.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:20 smithi148 ceph-mon[32311]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-04-23T18:36:21.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:20 smithi157 ceph-mon[32580]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-04-23T18:36:22.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:22 smithi132 ceph-mon[29178]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.3 KiB/s wr, 26 op/s 2024-04-23T18:36:23.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:22 smithi148 ceph-mon[32311]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.3 KiB/s wr, 26 op/s 2024-04-23T18:36:23.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:22 smithi157 ceph-mon[32580]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.3 KiB/s wr, 26 op/s 2024-04-23T18:36:24.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:24 smithi132 ceph-mon[29178]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-23T18:36:25.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:24 smithi148 ceph-mon[32311]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-23T18:36:25.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:24 smithi157 ceph-mon[32580]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-23T18:36:26.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:26 smithi132 ceph-mon[29178]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.4 KiB/s wr, 17 op/s 2024-04-23T18:36:27.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:26 smithi148 ceph-mon[32311]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.4 KiB/s wr, 17 op/s 2024-04-23T18:36:27.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:26 smithi157 ceph-mon[32580]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.4 KiB/s wr, 17 op/s 2024-04-23T18:36:27.342 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:36:27.342 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.45 2024-04-23T18:36:27.342 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:36:27.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.455+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1050812640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fb01010a0 msgr2=0x7f3fb0101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.455+0000 7f3fb55fe640 1 --2- 172.21.15.132:0/1050812640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fb01010a0 0x7f3fb0101480 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009900 tx=0x7f3fa002a410 comp rx=0 tx=0).stop 2024-04-23T18:36:27.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.456+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1050812640 shutdown_connections 2024-04-23T18:36:27.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.456+0000 7f3fb55fe640 1 --2- 172.21.15.132:0/1050812640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0101a50 0x7f3fb0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.456+0000 7f3fb55fe640 1 --2- 172.21.15.132:0/1050812640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb0106170 0x7f3fb010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.456+0000 7f3fb55fe640 1 --2- 172.21.15.132:0/1050812640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fb01010a0 0x7f3fb0101480 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.456+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1050812640 >> 172.21.15.132:0/1050812640 conn(0x7f3fb0077f50 msgr2=0x7f3fb0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:27.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.456+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1050812640 shutdown_connections 2024-04-23T18:36:27.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.456+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1050812640 wait complete. 2024-04-23T18:36:27.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.456+0000 7f3fb55fe640 1 Processor -- start 2024-04-23T18:36:27.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.456+0000 7f3fb55fe640 1 -- start start 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3fb55fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fb01010a0 0x7f3fb0102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3fb55fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0106170 0x7f3fb0103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3fb55fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb01035b0 0x7f3fb01a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3fb55fe640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01110d0 con 0x7f3fb01010a0 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3fb55fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0110f50 con 0x7f3fb01035b0 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3fb55fe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0111250 con 0x7f3fb0106170 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3faeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fb01010a0 0x7f3fb0102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3faeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fb01010a0 0x7f3fb0102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46226/0 (socket says 172.21.15.132:46226) 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3faeffd640 1 -- 172.21.15.132:0/1611795922 learned_addr learned my addr 172.21.15.132:0/1611795922 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3fae7fc640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0106170 0x7f3fb0103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3faf7fe640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb01035b0 0x7f3fb01a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3faeffd640 1 -- 172.21.15.132:0/1611795922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0106170 msgr2=0x7f3fb0103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3faeffd640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0106170 0x7f3fb0103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3faeffd640 1 -- 172.21.15.132:0/1611795922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb01035b0 msgr2=0x7f3fb01a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3faeffd640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb01035b0 0x7f3fb01a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3faeffd640 1 -- 172.21.15.132:0/1611795922 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb01a6fe0 con 0x7f3fb01010a0 2024-04-23T18:36:27.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3faeffd640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fb01010a0 0x7f3fb0102b30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0006010 tx=0x7f3fa0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:27.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3f97fff640 1 -- 172.21.15.132:0/1611795922 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa003f030 con 0x7f3fb01010a0 2024-04-23T18:36:27.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3f97fff640 1 -- 172.21.15.132:0/1611795922 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fa0003aa0 con 0x7f3fb01010a0 2024-04-23T18:36:27.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1611795922 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb01a7270 con 0x7f3fb01010a0 2024-04-23T18:36:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3f97fff640 1 -- 172.21.15.132:0/1611795922 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0035070 con 0x7f3fb01010a0 2024-04-23T18:36:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.457+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1611795922 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb01a7760 con 0x7f3fb01010a0 2024-04-23T18:36:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.458+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1611795922 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb0068d70 con 0x7f3fb01010a0 2024-04-23T18:36:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.460+0000 7f3f97fff640 1 -- 172.21.15.132:0/1611795922 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f3fa0042030 con 0x7f3fb01010a0 2024-04-23T18:36:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.460+0000 7f3f97fff640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f3f980a3e40 0x7f3f980a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.460+0000 7f3f97fff640 1 -- 172.21.15.132:0/1611795922 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f3fa00f1290 con 0x7f3fb01010a0 2024-04-23T18:36:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.461+0000 7f3fae7fc640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f3f980a3e40 0x7f3f980a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.461+0000 7f3fae7fc640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f3f980a3e40 0x7f3f980a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3fb01044d0 tx=0x7f3fa4005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:27.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.468+0000 7f3f97fff640 1 -- 172.21.15.132:0/1611795922 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fa00bd7a0 con 0x7f3fb01010a0 2024-04-23T18:36:27.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.736+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1611795922 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fb010a3b0 con 0x7f3fb01010a0 2024-04-23T18:36:27.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.736+0000 7f3f97fff640 1 -- 172.21.15.132:0/1611795922 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11946 (secure 0 0 0) 0x7f3fa00bd0a0 con 0x7f3fb01010a0 2024-04-23T18:36:27.729 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:27.729 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:36:01.455959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24460,24425,24394,24491,24515]},"id":1}]} 2024-04-23T18:36:27.729 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 26 2024-04-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.739+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1611795922 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f3f980a3e40 msgr2=0x7f3f980a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.739+0000 7f3fb55fe640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f3f980a3e40 0x7f3f980a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3fb01044d0 tx=0x7f3fa4005f90 comp rx=0 tx=0).stop 2024-04-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.739+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1611795922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fb01010a0 msgr2=0x7f3fb0102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.739+0000 7f3fb55fe640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fb01010a0 0x7f3fb0102b30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0006010 tx=0x7f3fa0004270 comp rx=0 tx=0).stop 2024-04-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.739+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1611795922 shutdown_connections 2024-04-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.739+0000 7f3fb55fe640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f3f980a3e40 0x7f3f980a6300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.739+0000 7f3fb55fe640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb0106170 0x7f3fb0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.739+0000 7f3fb55fe640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb01035b0 0x7f3fb01a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.739+0000 7f3fb55fe640 1 --2- 172.21.15.132:0/1611795922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fb01010a0 0x7f3fb0102b30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.739+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1611795922 >> 172.21.15.132:0/1611795922 conn(0x7f3fb0077f50 msgr2=0x7f3fb0069510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:27.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.739+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1611795922 shutdown_connections 2024-04-23T18:36:27.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.739+0000 7f3fb55fe640 1 -- 172.21.15.132:0/1611795922 wait complete. 2024-04-23T18:36:27.744 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24515 quiesce db / --timeout 60 --expiration 79.45 --await-for 42.0' 2024-04-23T18:36:27.744 DEBUG:teuthology.orchestra.run.smithi132:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24515 quiesce db / --timeout 60 --expiration 79.45 --await-for 42.0 2024-04-23T18:36:27.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.875+0000 7f2741aa7640 1 -- 172.21.15.132:0/1193510680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f273c101090 msgr2=0x7f273c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.875+0000 7f2741aa7640 1 --2- 172.21.15.132:0/1193510680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f273c101090 0x7f273c101470 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f272c00d910 tx=0x7f272c02a160 comp rx=0 tx=0).stop 2024-04-23T18:36:27.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.875+0000 7f2741aa7640 1 -- 172.21.15.132:0/1193510680 shutdown_connections 2024-04-23T18:36:27.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.875+0000 7f2741aa7640 1 --2- 172.21.15.132:0/1193510680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f273c106160 0x7f273c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.875+0000 7f2741aa7640 1 --2- 172.21.15.132:0/1193510680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f273c101a40 0x7f273c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.875+0000 7f2741aa7640 1 --2- 172.21.15.132:0/1193510680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f273c101090 0x7f273c101470 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.875+0000 7f2741aa7640 1 -- 172.21.15.132:0/1193510680 >> 172.21.15.132:0/1193510680 conn(0x7f273c0771d0 msgr2=0x7f273c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.875+0000 7f2741aa7640 1 -- 172.21.15.132:0/1193510680 shutdown_connections 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.875+0000 7f2741aa7640 1 -- 172.21.15.132:0/1193510680 wait complete. 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.876+0000 7f2741aa7640 1 Processor -- start 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.876+0000 7f2741aa7640 1 -- start start 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.876+0000 7f2741aa7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f273c101090 0x7f273c196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.876+0000 7f2741aa7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f273c106160 0x7f273c196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.876+0000 7f2741aa7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f273c19ae00 0x7f273c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.876+0000 7f2741aa7640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f273c10ffc0 con 0x7f273c106160 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.876+0000 7f2741aa7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f273c10fe40 con 0x7f273c101090 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.876+0000 7f2741aa7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f273c110140 con 0x7f273c19ae00 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.876+0000 7f273affd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f273c106160 0x7f273c196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.876+0000 7f273affd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f273c106160 0x7f273c196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46244/0 (socket says 172.21.15.132:46244) 2024-04-23T18:36:27.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.876+0000 7f273affd640 1 -- 172.21.15.132:0/319991990 learned_addr learned my addr 172.21.15.132:0/319991990 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:27.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.876+0000 7f273b7fe640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f273c101090 0x7f273c196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.877+0000 7f273affd640 1 -- 172.21.15.132:0/319991990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f273c19ae00 msgr2=0x7f273c19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.877+0000 7f273affd640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f273c19ae00 0x7f273c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.877+0000 7f273affd640 1 -- 172.21.15.132:0/319991990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f273c101090 msgr2=0x7f273c196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.877+0000 7f273affd640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f273c101090 0x7f273c196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.877+0000 7f273affd640 1 -- 172.21.15.132:0/319991990 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c19b8f0 con 0x7f273c106160 2024-04-23T18:36:27.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.877+0000 7f273affd640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f273c106160 0x7f273c196a80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f273000d770 tx=0x7f273000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:27.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.877+0000 7f2738ff9640 1 -- 172.21.15.132:0/319991990 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730017070 con 0x7f273c106160 2024-04-23T18:36:27.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.877+0000 7f2741aa7640 1 -- 172.21.15.132:0/319991990 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273c19bbe0 con 0x7f273c106160 2024-04-23T18:36:27.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.877+0000 7f2741aa7640 1 -- 172.21.15.132:0/319991990 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273c1a3420 con 0x7f273c106160 2024-04-23T18:36:27.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.877+0000 7f2738ff9640 1 -- 172.21.15.132:0/319991990 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2730009c30 con 0x7f273c106160 2024-04-23T18:36:27.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.877+0000 7f2738ff9640 1 -- 172.21.15.132:0/319991990 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273000f620 con 0x7f273c106160 2024-04-23T18:36:27.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.880+0000 7f2738ff9640 1 -- 172.21.15.132:0/319991990 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f273001a070 con 0x7f273c106160 2024-04-23T18:36:27.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.880+0000 7f2738ff9640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f27240a3ec0 0x7f27240a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.880+0000 7f2738ff9640 1 -- 172.21.15.132:0/319991990 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f27300d03b0 con 0x7f273c106160 2024-04-23T18:36:27.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.880+0000 7f27447c2740 1 Processor -- start 2024-04-23T18:36:27.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.880+0000 7f273b7fe640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f27240a3ec0 0x7f27240a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.880+0000 7f27447c2740 1 -- start start 2024-04-23T18:36:27.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f27447c2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a0d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f27447c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a20360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f27447c2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 0x55f1f2a22c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f27447c2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55f1f2972370 con 0x55f1f2a208a0 2024-04-23T18:36:27.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f27447c2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f1f290f2e0 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f27447c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f1f296e330 con 0x55f1f2a0d7a0 2024-04-23T18:36:27.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f273bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 0x55f1f2a22c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f273bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 0x55f1f2a22c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46252/0 (socket says 172.21.15.132:46252) 2024-04-23T18:36:27.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f273bfff640 1 -- 172.21.15.132:0/1381102155 learned_addr learned my addr 172.21.15.132:0/1381102155 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:27.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f273b7fe640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f27240a3ec0 0x7f27240a6380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f272c000c00 tx=0x7f272c02bad0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:27.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f273affd640 1 --2- 172.21.15.132:0/1381102155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a20360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f273b7fe640 1 --2- 172.21.15.132:0/1381102155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a0d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f273bfff640 1 -- 172.21.15.132:0/1381102155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 msgr2=0x55f1f2a20360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:27.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f273bfff640 1 --2- 172.21.15.132:0/1381102155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a20360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f273bfff640 1 -- 172.21.15.132:0/1381102155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a0cdf0 msgr2=0x55f1f2a0d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:27.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f273bfff640 1 --2- 172.21.15.132:0/1381102155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a0d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f273bfff640 1 -- 172.21.15.132:0/1381102155 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f2750d80 con 0x55f1f2a208a0 2024-04-23T18:36:27.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.881+0000 7f273b7fe640 1 --2- 172.21.15.132:0/1381102155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a0d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:27.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.882+0000 7f273bfff640 1 --2- 172.21.15.132:0/1381102155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 0x55f1f2a22c90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f273400de20 tx=0x7f273400d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d17ddfcdfa13dc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:27.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.882+0000 7f2721ffb640 1 -- 172.21.15.132:0/1381102155 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734019020 con 0x55f1f2a208a0 2024-04-23T18:36:27.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.882+0000 7f2721ffb640 1 -- 172.21.15.132:0/1381102155 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2734004d20 con 0x55f1f2a208a0 2024-04-23T18:36:27.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.882+0000 7f2721ffb640 1 -- 172.21.15.132:0/1381102155 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734015770 con 0x55f1f2a208a0 2024-04-23T18:36:27.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.883+0000 7f27447c2740 1 -- 172.21.15.132:0/1381102155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 msgr2=0x55f1f2a22c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.883+0000 7f27447c2740 1 --2- 172.21.15.132:0/1381102155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 0x55f1f2a22c90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f273400de20 tx=0x7f273400d2e0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.883+0000 7f27447c2740 1 -- 172.21.15.132:0/1381102155 shutdown_connections 2024-04-23T18:36:27.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.883+0000 7f27447c2740 1 --2- 172.21.15.132:0/1381102155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a20360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.883+0000 7f27447c2740 1 --2- 172.21.15.132:0/1381102155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 0x55f1f2a22c90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.883+0000 7f27447c2740 1 --2- 172.21.15.132:0/1381102155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a0d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.883+0000 7f27447c2740 1 -- 172.21.15.132:0/1381102155 >> 172.21.15.132:0/1381102155 conn(0x55f1f2a093c0 msgr2=0x55f1f2a097a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:27.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.883+0000 7f27447c2740 1 -- 172.21.15.132:0/1381102155 shutdown_connections 2024-04-23T18:36:27.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.883+0000 7f27447c2740 1 -- 172.21.15.132:0/1381102155 wait complete. 2024-04-23T18:36:27.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.883+0000 7f27447c2740 1 -- start start 2024-04-23T18:36:27.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.883+0000 7f27447c2740 1 Processor -- start 2024-04-23T18:36:27.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f27447c2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a1d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f27447c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a1dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f27447c2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a208a0 0x55f1f2a1e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f273b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a1d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f273b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a1d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46262/0 (socket says 172.21.15.132:46262) 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f273b7fe640 1 -- 172.21.15.132:0/1172985106 learned_addr learned my addr 172.21.15.132:0/1172985106 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f27447c2740 1 -- 172.21.15.132:0/1172985106 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55f1f2a23660 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f27447c2740 1 -- 172.21.15.132:0/1172985106 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f1f290cfb0 con 0x55f1f2a208a0 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f27447c2740 1 -- 172.21.15.132:0/1172985106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f1f2a237e0 con 0x55f1f2a0d7a0 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f273affd640 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a1dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f27447c2740 10 client.0 _ll_register_callbacks cb 0x55f1f290c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f273bfff640 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a208a0 0x55f1f2a1e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f273affd640 1 -- 172.21.15.132:0/1172985106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a208a0 msgr2=0x55f1f2a1e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f273affd640 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a208a0 0x55f1f2a1e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f273affd640 1 -- 172.21.15.132:0/1172985106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 msgr2=0x55f1f2a1d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.884+0000 7f273affd640 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a1d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.885+0000 7f273affd640 1 -- 172.21.15.132:0/1172985106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f27b1510 con 0x55f1f2a0d7a0 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.885+0000 7f273b7fe640 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a1d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.885+0000 7f273affd640 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a1dd20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27300cd970 tx=0x7f2730009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d464b70b6ecb362d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.885+0000 7f271affd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.885+0000 7f271affd640 1 -- 172.21.15.132:0/1172985106 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27300b5060 con 0x55f1f2a0d7a0 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.885+0000 7f271affd640 1 -- 172.21.15.132:0/1172985106 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f27300cdc50 con 0x55f1f2a0d7a0 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.885+0000 7f27447c2740 1 -- 172.21.15.132:0/1172985106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2734004d20 con 0x55f1f2a0d7a0 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.885+0000 7f271affd640 1 -- 172.21.15.132:0/1172985106 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2730021040 con 0x55f1f2a0d7a0 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.885+0000 7f27447c2740 10 client.25153 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.885+0000 7f27447c2740 1 -- 172.21.15.132:0/1172985106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1f2644670 con 0x55f1f2a0d7a0 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.886+0000 7f271affd640 1 -- 172.21.15.132:0/1172985106 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2734004d20 con 0x55f1f2a0d7a0 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.886+0000 7f271affd640 1 -- 172.21.15.132:0/1172985106 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f27300b0cb0 con 0x55f1f2a0d7a0 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.886+0000 7f27447c2740 10 client.25153 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.886+0000 7f27447c2740 10 client.25153 resolve_mds: validated gid 24515 aka daemon mds.b 2024-04-23T18:36:27.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.886+0000 7f27447c2740 20 client.25153 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.886+0000 7f27447c2740 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55f1f2a30b60 0x55f1f2a32fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.886+0000 7f27447c2740 4 client.25153 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.886+0000 7f27447c2740 1 -- 172.21.15.132:0/1172985106 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1f297fd90 con 0x55f1f2a30b60 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.886+0000 7f273b7fe640 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55f1f2a30b60 0x55f1f2a32fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.887+0000 7f271affd640 10 client.25153 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.887+0000 7f273b7fe640 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55f1f2a30b60 0x55f1f2a32fb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d317704d834187eb server_cookie=ba8f64d94e113fed in_seq=0 out_seq=0 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.887+0000 7f271affd640 1 -- 172.21.15.132:0/1172985106 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1f297fd90 con 0x55f1f2a30b60 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.887+0000 7f271affd640 10 client.25153 handle_command_reply: tid=0 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.887+0000 7f27447c2740 1 client.25153 shutdown 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.887+0000 7f27447c2740 1 -- 172.21.15.132:0/1172985106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 msgr2=0x55f1f2a1dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.887+0000 7f27447c2740 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a1dd20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27300cd970 tx=0x7f2730009f20 comp rx=0 tx=0).stop 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.888+0000 7f27447c2740 1 -- 172.21.15.132:0/1172985106 shutdown_connections 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.888+0000 7f27447c2740 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55f1f2a30b60 0x55f1f2a32fb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.888+0000 7f27447c2740 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a1dd20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.888+0000 7f271affd640 0 client.25153 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:27.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.888+0000 7f27447c2740 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a208a0 0x55f1f2a1e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.888+0000 7f27447c2740 1 --2- 172.21.15.132:0/1172985106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a1d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.888+0000 7f27447c2740 1 -- 172.21.15.132:0/1172985106 >> 172.21.15.132:0/1172985106 conn(0x55f1f2a093c0 msgr2=0x55f1f2a0f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:27.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.888+0000 7f27447c2740 1 -- 172.21.15.132:0/1172985106 shutdown_connections 2024-04-23T18:36:27.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.888+0000 7f27447c2740 1 -- 172.21.15.132:0/1172985106 wait complete. 2024-04-23T18:36:27.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.888+0000 7f27447c2740 20 client.25153 trim_cache size 0 max 16384 2024-04-23T18:36:27.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.905+0000 7f27447c2740 1 Processor -- start 2024-04-23T18:36:27.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.905+0000 7f27447c2740 1 -- start start 2024-04-23T18:36:27.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f27447c2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a2c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f27447c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a38880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f27447c2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a208a0 0x55f1f2a38dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f27447c2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f272800e7d0 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f27447c2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2728010310 con 0x55f1f2a208a0 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f27447c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f272800eea0 con 0x55f1f2a0d7a0 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f273b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a2c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f273b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a2c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46274/0 (socket says 172.21.15.132:46274) 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f273b7fe640 1 -- 172.21.15.132:0/940371409 learned_addr learned my addr 172.21.15.132:0/940371409 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f273bfff640 1 --2- 172.21.15.132:0/940371409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a208a0 0x55f1f2a38dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f273affd640 1 --2- 172.21.15.132:0/940371409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a38880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f273b7fe640 1 -- 172.21.15.132:0/940371409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 msgr2=0x55f1f2a38880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f273b7fe640 1 --2- 172.21.15.132:0/940371409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a38880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f273b7fe640 1 -- 172.21.15.132:0/940371409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a208a0 msgr2=0x55f1f2a38dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f273b7fe640 1 --2- 172.21.15.132:0/940371409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a208a0 0x55f1f2a38dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.906+0000 7f273b7fe640 1 -- 172.21.15.132:0/940371409 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f2a1bd70 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.907+0000 7f273b7fe640 1 --2- 172.21.15.132:0/940371409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a2c5f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f272c009230 tx=0x7f272c03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761d6fe9b4c8d90b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:27.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.907+0000 7f27217fa640 1 -- 172.21.15.132:0/940371409 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c005300 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.907+0000 7f27217fa640 1 -- 172.21.15.132:0/940371409 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f272c054070 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.907+0000 7f27217fa640 1 -- 172.21.15.132:0/940371409 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c03e6c0 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.908+0000 7f27447c2740 1 -- 172.21.15.132:0/940371409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 msgr2=0x55f1f2a2c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.908+0000 7f27447c2740 1 --2- 172.21.15.132:0/940371409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a2c5f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f272c009230 tx=0x7f272c03b000 comp rx=0 tx=0).stop 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.908+0000 7f27447c2740 1 -- 172.21.15.132:0/940371409 shutdown_connections 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.908+0000 7f27447c2740 1 --2- 172.21.15.132:0/940371409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a38880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.908+0000 7f27447c2740 1 --2- 172.21.15.132:0/940371409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a208a0 0x55f1f2a38dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.908+0000 7f27447c2740 1 --2- 172.21.15.132:0/940371409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a2c5f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.908+0000 7f27447c2740 1 -- 172.21.15.132:0/940371409 >> 172.21.15.132:0/940371409 conn(0x55f1f2a093c0 msgr2=0x55f1f2a0f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.908+0000 7f27447c2740 1 -- 172.21.15.132:0/940371409 shutdown_connections 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.908+0000 7f27447c2740 1 -- 172.21.15.132:0/940371409 wait complete. 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.908+0000 7f27447c2740 1 -- start start 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.908+0000 7f27447c2740 1 Processor -- start 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f27447c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a1a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f27447c2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a1ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f27447c2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 0x55f1f2a1b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f273bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 0x55f1f2a1b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f273bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 0x55f1f2a1b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46288/0 (socket says 172.21.15.132:46288) 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f273bfff640 1 -- 172.21.15.132:0/1111785921 learned_addr learned my addr 172.21.15.132:0/1111785921 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f27447c2740 1 -- 172.21.15.132:0/1111785921 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f272800fc40 con 0x55f1f2a208a0 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f27447c2740 1 -- 172.21.15.132:0/1111785921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f272800f570 con 0x55f1f2a0d7a0 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f27447c2740 1 -- 172.21.15.132:0/1111785921 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f272800c7b0 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f273b7fe640 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a1a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f273affd640 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a1ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f273b7fe640 1 -- 172.21.15.132:0/1111785921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a0d7a0 msgr2=0x55f1f2a1ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f273b7fe640 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a1ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.909+0000 7f273b7fe640 1 -- 172.21.15.132:0/1111785921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 msgr2=0x55f1f2a1b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.910+0000 7f273b7fe640 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 0x55f1f2a1b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.910+0000 7f273b7fe640 1 -- 172.21.15.132:0/1111785921 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f2a962f0 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.910+0000 7f273bfff640 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 0x55f1f2a1b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.910+0000 7f27447c2740 10 client.0 _ll_register_callbacks cb 0x7f273009ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.910+0000 7f27447c2740 1 -- 172.21.15.132:0/1111785921 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f272c054070 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.910+0000 7f273b7fe640 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a1a6e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f272c051040 tx=0x7f272c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fb69982bb56db76 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.910+0000 7f271affd640 10 client.25159 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.910+0000 7f271affd640 1 -- 172.21.15.132:0/1111785921 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c05c020 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.910+0000 7f271affd640 1 -- 172.21.15.132:0/1111785921 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f272c0053c0 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.910+0000 7f271affd640 1 -- 172.21.15.132:0/1111785921 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f272c0055c0 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.910+0000 7f27447c2740 10 client.25159 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.910+0000 7f27447c2740 1 -- 172.21.15.132:0/1111785921 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55f1f2a97380 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.911+0000 7f271affd640 1 -- 172.21.15.132:0/1111785921 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f272c047040 con 0x55f1f2a0cdf0 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.911+0000 7f27447c2740 10 client.25159 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:27.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.911+0000 7f27447c2740 10 client.25159 resolve_mds: validated gid 24515 aka daemon mds.b 2024-04-23T18:36:27.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.911+0000 7f27447c2740 20 client.25159 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:27.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.911+0000 7f27447c2740 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55f1f2a97a10 0x55f1f2a99eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:27.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.911+0000 7f27447c2740 4 client.25159 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.45, "await_for": 42.0}] 2024-04-23T18:36:27.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.911+0000 7f27447c2740 1 -- 172.21.15.132:0/1111785921 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.45, "await_for": 42.0}) -- 0x55f1f2979c00 con 0x55f1f2a97a10 2024-04-23T18:36:27.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.911+0000 7f273affd640 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55f1f2a97a10 0x55f1f2a99eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:27.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.911+0000 7f271affd640 10 client.25159 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:27.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.911+0000 7f273affd640 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55f1f2a97a10 0x55f1f2a99eb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=634f6ea05a93885c server_cookie=b29f1252885e47c2 in_seq=0 out_seq=0 2024-04-23T18:36:27.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.924+0000 7f271affd640 1 -- 172.21.15.132:0/1111785921 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+586 (crc 0 0 0) 0x55f1f2979c00 con 0x55f1f2a97a10 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.924+0000 7f271affd640 10 client.25159 handle_command_reply: tid=0 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.924+0000 7f27447c2740 1 client.25159 shutdown 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.924+0000 7f27447c2740 1 -- 172.21.15.132:0/1111785921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0cdf0 msgr2=0x55f1f2a1a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.924+0000 7f27447c2740 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a1a6e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f272c051040 tx=0x7f272c009260 comp rx=0 tx=0).stop 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.924+0000 7f27447c2740 1 -- 172.21.15.132:0/1111785921 shutdown_connections 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.924+0000 7f27447c2740 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55f1f2a97a10 0x55f1f2a99eb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.924+0000 7f27447c2740 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1f2a208a0 0x55f1f2a1b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.924+0000 7f27447c2740 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1f2a0d7a0 0x55f1f2a1ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f27447c2740 1 --2- 172.21.15.132:0/1111785921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f1f2a0cdf0 0x55f1f2a1a6e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f27447c2740 1 -- 172.21.15.132:0/1111785921 >> 172.21.15.132:0/1111785921 conn(0x55f1f2a093c0 msgr2=0x55f1f2a0e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f271affd640 0 client.25159 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f27447c2740 1 -- 172.21.15.132:0/1111785921 shutdown_connections 2024-04-23T18:36:27.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f27447c2740 1 -- 172.21.15.132:0/1111785921 wait complete. 2024-04-23T18:36:27.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f27447c2740 20 client.25159 trim_cache size 0 max 16384 2024-04-23T18:36:27.918 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:36:27.918 INFO:teuthology.orchestra.run.smithi132.stdout: "epoch": 26, 2024-04-23T18:36:27.918 INFO:teuthology.orchestra.run.smithi132.stdout: "leader": 24515, 2024-04-23T18:36:27.918 INFO:teuthology.orchestra.run.smithi132.stdout: "set_version": 2, 2024-04-23T18:36:27.918 INFO:teuthology.orchestra.run.smithi132.stdout: "sets": { 2024-04-23T18:36:27.918 INFO:teuthology.orchestra.run.smithi132.stdout: "a559ed": { 2024-04-23T18:36:27.918 INFO:teuthology.orchestra.run.smithi132.stdout: "version": 2, 2024-04-23T18:36:27.918 INFO:teuthology.orchestra.run.smithi132.stdout: "age_ref": 0.0, 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: "state": { 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "QUIESCED", 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: "age": 0.0 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: "timeout": 60.0, 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: "expiration": 79.5, 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: "members": { 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: "file:/": { 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: "excluded": false, 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: "state": { 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "QUIESCED", 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: "age": 0.0 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:27.919 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:27.920 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:27.920 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:27.920 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:36:27.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f2741aa7640 1 -- 172.21.15.132:0/319991990 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f27240a3ec0 msgr2=0x7f27240a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f2741aa7640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f27240a3ec0 0x7f27240a6380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f272c000c00 tx=0x7f272c02bad0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f2741aa7640 1 -- 172.21.15.132:0/319991990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f273c106160 msgr2=0x7f273c196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f2741aa7640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f273c106160 0x7f273c196a80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f273000d770 tx=0x7f273000dc30 comp rx=0 tx=0).stop 2024-04-23T18:36:27.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f2741aa7640 1 -- 172.21.15.132:0/319991990 shutdown_connections 2024-04-23T18:36:27.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f2741aa7640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f27240a3ec0 0x7f27240a6380 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f2741aa7640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f273c19ae00 0x7f273c19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f2741aa7640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f273c106160 0x7f273c196a80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f2741aa7640 1 --2- 172.21.15.132:0/319991990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f273c101090 0x7f273c196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.925+0000 7f2741aa7640 1 -- 172.21.15.132:0/319991990 >> 172.21.15.132:0/319991990 conn(0x7f273c0771d0 msgr2=0x7f273c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:27.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.926+0000 7f2741aa7640 1 -- 172.21.15.132:0/319991990 shutdown_connections 2024-04-23T18:36:27.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:27.926+0000 7f2741aa7640 1 -- 172.21.15.132:0/319991990 wait complete. 2024-04-23T18:36:27.929 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a559ed', quiesce took 0.6 seconds. Will release after: 19.45 2024-04-23T18:36:28.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:28 smithi132 ceph-mon[29178]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-23T18:36:28.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:28 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1611795922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:29.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:28 smithi148 ceph-mon[32311]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-23T18:36:29.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:28 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1611795922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:29.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:28 smithi157 ceph-mon[32580]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-23T18:36:29.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:28 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1611795922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:30.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:30 smithi132 ceph-mon[29178]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-23T18:36:31.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:30 smithi148 ceph-mon[32311]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-23T18:36:31.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:30 smithi157 ceph-mon[32580]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-23T18:36:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.707+0000 7f8fc310f640 1 -- 172.21.15.132:0/1837295516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fbc106190 msgr2=0x7f8fbc10dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.707+0000 7f8fc310f640 1 --2- 172.21.15.132:0/1837295516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fbc106190 0x7f8fbc10dd60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009ca0 tx=0x7f8fb402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.707+0000 7f8fc310f640 1 -- 172.21.15.132:0/1837295516 shutdown_connections 2024-04-23T18:36:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.707+0000 7f8fc310f640 1 --2- 172.21.15.132:0/1837295516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fbc101a70 0x7f8fbc105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.707+0000 7f8fc310f640 1 --2- 172.21.15.132:0/1837295516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fbc106190 0x7f8fbc10dd60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.707+0000 7f8fc310f640 1 --2- 172.21.15.132:0/1837295516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fbc1010c0 0x7f8fbc1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.707+0000 7f8fc310f640 1 -- 172.21.15.132:0/1837295516 >> 172.21.15.132:0/1837295516 conn(0x7f8fbc077f70 msgr2=0x7f8fbc078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.707+0000 7f8fc310f640 1 -- 172.21.15.132:0/1837295516 shutdown_connections 2024-04-23T18:36:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.707+0000 7f8fc310f640 1 -- 172.21.15.132:0/1837295516 wait complete. 2024-04-23T18:36:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.708+0000 7f8fc310f640 1 Processor -- start 2024-04-23T18:36:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.708+0000 7f8fc310f640 1 -- start start 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc310f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fbc1010c0 0x7f8fbc1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc310f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fbc106190 0x7f8fbc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc310f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fbc19beb0 0x7f8fbc19c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc310f640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc111150 con 0x7f8fbc1010c0 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc310f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc110fd0 con 0x7f8fbc106190 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc310f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1112d0 con 0x7f8fbc19beb0 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc0eab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fbc1010c0 0x7f8fbc1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc0eab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fbc1010c0 0x7f8fbc1975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59368/0 (socket says 172.21.15.132:59368) 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc0eab640 1 -- 172.21.15.132:0/1373778139 learned_addr learned my addr 172.21.15.132:0/1373778139 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc0eab640 1 -- 172.21.15.132:0/1373778139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fbc19beb0 msgr2=0x7f8fbc19c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc0eab640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fbc19beb0 0x7f8fbc19c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc0eab640 1 -- 172.21.15.132:0/1373778139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fbc106190 msgr2=0x7f8fbc197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fbbfff640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fbc106190 0x7f8fbc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc0eab640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fbc106190 0x7f8fbc197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc0eab640 1 -- 172.21.15.132:0/1373778139 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc19c9a0 con 0x7f8fbc1010c0 2024-04-23T18:36:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fbbfff640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fbc106190 0x7f8fbc197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fc0eab640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fbc1010c0 0x7f8fbc1975f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000e9f0 tx=0x7f8fb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.709+0000 7f8fb9ffb640 1 -- 172.21.15.132:0/1373778139 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000cd80 con 0x7f8fbc1010c0 2024-04-23T18:36:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.710+0000 7f8fc310f640 1 -- 172.21.15.132:0/1373778139 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc19cc90 con 0x7f8fbc1010c0 2024-04-23T18:36:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.710+0000 7f8fb9ffb640 1 -- 172.21.15.132:0/1373778139 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb00108e0 con 0x7f8fbc1010c0 2024-04-23T18:36:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.710+0000 7f8fb9ffb640 1 -- 172.21.15.132:0/1373778139 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0018980 con 0x7f8fbc1010c0 2024-04-23T18:36:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.710+0000 7f8fc310f640 1 -- 172.21.15.132:0/1373778139 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc06a1f0 con 0x7f8fbc1010c0 2024-04-23T18:36:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.713+0000 7f8fc5e22740 1 Processor -- start 2024-04-23T18:36:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.713+0000 7f8fb9ffb640 1 -- 172.21.15.132:0/1373778139 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f8fb0010430 con 0x7f8fbc1010c0 2024-04-23T18:36:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.713+0000 7f8fc5e22740 1 -- start start 2024-04-23T18:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fc5e22740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1f6c0 0x55d2fab1faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fc5e22740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1b130 0x55d2fab1b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fc5e22740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 0x55d2fab27610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fc5e22740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55d2faa76970 con 0x55d2fab1baf0 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fc5e22740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d2faa134a0 con 0x55d2fab1b130 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fc5e22740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d2faa72930 con 0x55d2fab1f6c0 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fbbfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 0x55d2fab27610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fbbfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 0x55d2fab27610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59374/0 (socket says 172.21.15.132:59374) 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fbbfff640 1 -- 172.21.15.132:0/2744428504 learned_addr learned my addr 172.21.15.132:0/2744428504 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/2744428504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1f6c0 0x55d2fab1faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fbbfff640 1 -- 172.21.15.132:0/2744428504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1f6c0 msgr2=0x55d2fab1faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fc0eab640 1 --2- 172.21.15.132:0/2744428504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1b130 0x55d2fab1b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fbbfff640 1 --2- 172.21.15.132:0/2744428504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1f6c0 0x55d2fab1faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fbbfff640 1 -- 172.21.15.132:0/2744428504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1b130 msgr2=0x55d2fab1b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fbbfff640 1 --2- 172.21.15.132:0/2744428504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1b130 0x55d2fab1b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fbbfff640 1 -- 172.21.15.132:0/2744428504 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2fa789210 con 0x55d2fab1baf0 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.714+0000 7f8fc0eab640 1 --2- 172.21.15.132:0/2744428504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1b130 0x55d2fab1b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.715+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/2744428504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1f6c0 0x55d2fab1faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.715+0000 7f8fbbfff640 1 --2- 172.21.15.132:0/2744428504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 0x55d2fab27610 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009c20 tx=0x7f8fa8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf480169238c3665 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.715+0000 7f8fa6ffd640 1 -- 172.21.15.132:0/2744428504 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8004030 con 0x55d2fab1baf0 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.715+0000 7f8fa6ffd640 1 -- 172.21.15.132:0/2744428504 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fa80035f0 con 0x55d2fab1baf0 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.715+0000 7f8fa6ffd640 1 -- 172.21.15.132:0/2744428504 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8012670 con 0x55d2fab1baf0 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.715+0000 7f8fb9ffb640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8fac0a3fe0 0x7f8fac0a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.715+0000 7f8fb9ffb640 1 -- 172.21.15.132:0/1373778139 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f8fb00d0f50 con 0x7f8fbc1010c0 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.716+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8fac0a3fe0 0x7f8fac0a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.716+0000 7f8fc5e22740 1 -- 172.21.15.132:0/2744428504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 msgr2=0x55d2fab27610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.716+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/2744428504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 0x55d2fab27610 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009c20 tx=0x7f8fa8010740 comp rx=0 tx=0).stop 2024-04-23T18:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.716+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8fac0a3fe0 0x7f8fac0a64a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8fb402f540 tx=0x7f8fb4005df0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.716+0000 7f8fc5e22740 1 -- 172.21.15.132:0/2744428504 shutdown_connections 2024-04-23T18:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.716+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/2744428504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 0x55d2fab27610 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.716+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/2744428504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1b130 0x55d2fab1b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.716+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/2744428504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1f6c0 0x55d2fab1faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.716+0000 7f8fc5e22740 1 -- 172.21.15.132:0/2744428504 >> 172.21.15.132:0/2744428504 conn(0x55d2fab0d5e0 msgr2=0x55d2fab0d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.717+0000 7f8fc5e22740 1 -- 172.21.15.132:0/2744428504 shutdown_connections 2024-04-23T18:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.717+0000 7f8fc5e22740 1 -- 172.21.15.132:0/2744428504 wait complete. 2024-04-23T18:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.717+0000 7f8fc5e22740 1 -- start start 2024-04-23T18:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.717+0000 7f8fc5e22740 1 Processor -- start 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc5e22740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1b130 0x55d2fab27320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc5e22740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1baf0 0x55d2fab24610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc5e22740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1f6c0 0x55d2fab24b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc5e22740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55d2fab27c10 con 0x55d2fab1f6c0 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc5e22740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d2faa110b0 con 0x55d2fab1b130 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc5e22740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d2fab27d90 con 0x55d2fab1baf0 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc5e22740 10 client.0 _ll_register_callbacks cb 0x55d2fa9fed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc16ac640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1f6c0 0x55d2fab24b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc16ac640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1f6c0 0x55d2fab24b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59390/0 (socket says 172.21.15.132:59390) 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc16ac640 1 -- 172.21.15.132:0/1443266215 learned_addr learned my addr 172.21.15.132:0/1443266215 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fbbfff640 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1baf0 0x55d2fab24610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc0eab640 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1b130 0x55d2fab27320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc16ac640 1 -- 172.21.15.132:0/1443266215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1baf0 msgr2=0x55d2fab24610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1baf0 0x55d2fab24610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc16ac640 1 -- 172.21.15.132:0/1443266215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1b130 msgr2=0x55d2fab27320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1b130 0x55d2fab27320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fc16ac640 1 -- 172.21.15.132:0/1443266215 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2fa855620 con 0x55d2fab1f6c0 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.718+0000 7f8fbbfff640 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1baf0 0x55d2fab24610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.719+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1f6c0 0x55d2fab24b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009a60 tx=0x7f8fb4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683201d9b568887e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.719+0000 7f8f9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.719+0000 7f8f9bfff640 1 -- 172.21.15.132:0/1443266215 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4004450 con 0x55d2fab1f6c0 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.719+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1443266215 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fa80035f0 con 0x55d2fab1f6c0 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.719+0000 7f8f9bfff640 1 -- 172.21.15.132:0/1443266215 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f8fb40045f0 con 0x55d2fab1f6c0 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.719+0000 7f8f9bfff640 1 -- 172.21.15.132:0/1443266215 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8fb40047f0 con 0x55d2fab1f6c0 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.719+0000 7f8fc5e22740 10 client.15726 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.719+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1443266215 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2fab339e0 con 0x55d2fab1f6c0 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.719+0000 7f8f9bfff640 1 -- 172.21.15.132:0/1443266215 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fa80035f0 con 0x55d2fab1f6c0 2024-04-23T18:36:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.720+0000 7f8f9bfff640 1 -- 172.21.15.132:0/1443266215 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8fb4030780 con 0x55d2fab1f6c0 2024-04-23T18:36:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.720+0000 7f8fc5e22740 10 client.15726 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.720+0000 7f8fc5e22740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.720+0000 7f8fc5e22740 20 client.15726 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.720+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55d2fab34ea0 0x55d2fab37290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.720+0000 7f8fc5e22740 4 client.15726 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.720+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1443266215 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2faa84390 con 0x55d2fab34ea0 2024-04-23T18:36:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.721+0000 7f8fc0eab640 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55d2fab34ea0 0x55d2fab37290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.721+0000 7f8f9bfff640 10 client.15726 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.721+0000 7f8fc0eab640 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55d2fab34ea0 0x55d2fab37290 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe95a13e28c76ed8 server_cookie=1bf5c7b45a3d7602 in_seq=0 out_seq=0 2024-04-23T18:36:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8f9bfff640 1 -- 172.21.15.132:0/1443266215 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2faa84390 con 0x55d2fab34ea0 2024-04-23T18:36:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8f9bfff640 10 client.15726 handle_command_reply: tid=0 2024-04-23T18:36:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8fc5e22740 1 client.15726 shutdown 2024-04-23T18:36:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1443266215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1f6c0 msgr2=0x55d2fab24b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1f6c0 0x55d2fab24b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009a60 tx=0x7f8fb4005d80 comp rx=0 tx=0).stop 2024-04-23T18:36:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1443266215 shutdown_connections 2024-04-23T18:36:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55d2fab34ea0 0x55d2fab37290 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1baf0 0x55d2fab24610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1f6c0 0x55d2fab24b50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1443266215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1b130 0x55d2fab27320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1443266215 >> 172.21.15.132:0/1443266215 conn(0x55d2fab0d5e0 msgr2=0x55d2fab1a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8f9bfff640 0 client.15726 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1443266215 shutdown_connections 2024-04-23T18:36:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1443266215 wait complete. 2024-04-23T18:36:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.722+0000 7f8fc5e22740 20 client.15726 trim_cache size 0 max 16384 2024-04-23T18:36:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fc5e22740 1 Processor -- start 2024-04-23T18:36:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fc5e22740 1 -- start start 2024-04-23T18:36:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fc5e22740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1b130 0x55d2fab30b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fc5e22740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 0x55d2fab3cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fc5e22740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1f6c0 0x55d2fab3d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fc5e22740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1ab740 con 0x55d2fab1baf0 2024-04-23T18:36:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fc5e22740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1ad280 con 0x55d2fab1f6c0 2024-04-23T18:36:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fc5e22740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1abe10 con 0x55d2fab1b130 2024-04-23T18:36:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fbbfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 0x55d2fab3cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fbbfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 0x55d2fab3cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59396/0 (socket says 172.21.15.132:59396) 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fbbfff640 1 -- 172.21.15.132:0/1904969307 learned_addr learned my addr 172.21.15.132:0/1904969307 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fc0eab640 1 --2- 172.21.15.132:0/1904969307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1b130 0x55d2fab30b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.751+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/1904969307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1f6c0 0x55d2fab3d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.752+0000 7f8fc16ac640 1 -- 172.21.15.132:0/1904969307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1b130 msgr2=0x55d2fab30b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.752+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/1904969307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1b130 0x55d2fab30b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.752+0000 7f8fc16ac640 1 -- 172.21.15.132:0/1904969307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 msgr2=0x55d2fab3cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.752+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/1904969307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 0x55d2fab3cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.752+0000 7f8fc16ac640 1 -- 172.21.15.132:0/1904969307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2fab114d0 con 0x55d2fab1f6c0 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.752+0000 7f8fbbfff640 1 --2- 172.21.15.132:0/1904969307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 0x55d2fab3cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.752+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/1904969307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1f6c0 0x55d2fab3d0e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009bf0 tx=0x7f8fb4004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee9f50b1005bad11 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.752+0000 7f8fa67fc640 1 -- 172.21.15.132:0/1904969307 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb405c070 con 0x55d2fab1f6c0 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.752+0000 7f8fa67fc640 1 -- 172.21.15.132:0/1904969307 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb4036050 con 0x55d2fab1f6c0 2024-04-23T18:36:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.752+0000 7f8fa67fc640 1 -- 172.21.15.132:0/1904969307 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb402f610 con 0x55d2fab1f6c0 2024-04-23T18:36:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.753+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1904969307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1f6c0 msgr2=0x55d2fab3d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.753+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1904969307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1f6c0 0x55d2fab3d0e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009bf0 tx=0x7f8fb4004990 comp rx=0 tx=0).stop 2024-04-23T18:36:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.753+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1904969307 shutdown_connections 2024-04-23T18:36:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.753+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1904969307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1f6c0 0x55d2fab3d0e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.753+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1904969307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1baf0 0x55d2fab3cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.753+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1904969307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1b130 0x55d2fab30b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.753+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1904969307 >> 172.21.15.132:0/1904969307 conn(0x55d2fab0d5e0 msgr2=0x55d2fab20360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.753+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1904969307 shutdown_connections 2024-04-23T18:36:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.753+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1904969307 wait complete. 2024-04-23T18:36:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.754+0000 7f8fc5e22740 1 -- start start 2024-04-23T18:36:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.754+0000 7f8fc5e22740 1 Processor -- start 2024-04-23T18:36:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.754+0000 7f8fc5e22740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1b130 0x55d2fab263f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.754+0000 7f8fc5e22740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1baf0 0x55d2fab26930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.754+0000 7f8fc5e22740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1f6c0 0x55d2fab26e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.754+0000 7f8fc5e22740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1acbb0 con 0x55d2fab1b130 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.754+0000 7f8fc5e22740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1ac4e0 con 0x55d2fab1baf0 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.754+0000 7f8fc5e22740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1a97d0 con 0x55d2fab1f6c0 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.754+0000 7f8fc5e22740 10 client.0 _ll_register_callbacks cb 0x7f8fb4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.754+0000 7f8fc0eab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1b130 0x55d2fab263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.754+0000 7f8fc0eab640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1b130 0x55d2fab263f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59402/0 (socket says 172.21.15.132:59402) 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8fc0eab640 1 -- 172.21.15.132:0/1402960378 learned_addr learned my addr 172.21.15.132:0/1402960378 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1f6c0 0x55d2fab26e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8fbbfff640 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1baf0 0x55d2fab26930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8fc0eab640 1 -- 172.21.15.132:0/1402960378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1f6c0 msgr2=0x55d2fab26e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8fc0eab640 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1f6c0 0x55d2fab26e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8fc0eab640 1 -- 172.21.15.132:0/1402960378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1baf0 msgr2=0x55d2fab26930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8fc0eab640 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1baf0 0x55d2fab26930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8fc0eab640 1 -- 172.21.15.132:0/1402960378 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2fab092e0 con 0x55d2fab1b130 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8fc16ac640 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1f6c0 0x55d2fab26e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8fc0eab640 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1b130 0x55d2fab263f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0099600 tx=0x7f8fb001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2125953fd04120f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8f9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8f9bfff640 1 -- 172.21.15.132:0/1402960378 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00b5070 con 0x55d2fab1b130 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8f9bfff640 1 -- 172.21.15.132:0/1402960378 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f8fb0010a90 con 0x55d2fab1b130 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.755+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1402960378 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb4036050 con 0x55d2fab1b130 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.756+0000 7f8f9bfff640 1 -- 172.21.15.132:0/1402960378 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8fb0087040 con 0x55d2fab1b130 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.756+0000 7f8fc5e22740 10 client.15732 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.756+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1402960378 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2fab9b690 con 0x55d2fab1b130 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.756+0000 7f8f9bfff640 1 -- 172.21.15.132:0/1402960378 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb4036050 con 0x55d2fab1b130 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.756+0000 7f8f9bfff640 1 -- 172.21.15.132:0/1402960378 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8fb00ce790 con 0x55d2fab1b130 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.756+0000 7f8fc5e22740 10 client.15732 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.756+0000 7f8fc5e22740 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.757+0000 7f8fc5e22740 20 client.15732 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.757+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55d2fab9bd40 0x55d2fab9e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.757+0000 7f8fc5e22740 4 client.15732 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:36:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.757+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1402960378 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2faa7e200 con 0x55d2fab9bd40 2024-04-23T18:36:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.757+0000 7f8fbbfff640 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55d2fab9bd40 0x55d2fab9e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.757+0000 7f8f9bfff640 10 client.15732 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.757+0000 7f8fbbfff640 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55d2fab9bd40 0x55d2fab9e160 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a2ebfb81ddb438 server_cookie=d07595986d8ef63 in_seq=0 out_seq=0 2024-04-23T18:36:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.764+0000 7f8f9bfff640 1 -- 172.21.15.132:0/1402960378 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2faa7e200 con 0x55d2fab9bd40 2024-04-23T18:36:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.764+0000 7f8f9bfff640 10 client.15732 handle_command_reply: tid=0 2024-04-23T18:36:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.764+0000 7f8fc5e22740 1 client.15732 shutdown 2024-04-23T18:36:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.765+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1402960378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1b130 msgr2=0x55d2fab263f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.765+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1b130 0x55d2fab263f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0099600 tx=0x7f8fb001f000 comp rx=0 tx=0).stop 2024-04-23T18:36:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.765+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1402960378 shutdown_connections 2024-04-23T18:36:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.765+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55d2fab9bd40 0x55d2fab9e160 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.765+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2fab1f6c0 0x55d2fab26e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.765+0000 7f8f9bfff640 0 client.15732 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.765+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d2fab1baf0 0x55d2fab26930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.765+0000 7f8fc5e22740 1 --2- 172.21.15.132:0/1402960378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2fab1b130 0x55d2fab263f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.765+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1402960378 >> 172.21.15.132:0/1402960378 conn(0x55d2fab0d5e0 msgr2=0x55d2fab245d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.765+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1402960378 shutdown_connections 2024-04-23T18:36:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.765+0000 7f8fc5e22740 1 -- 172.21.15.132:0/1402960378 wait complete. 2024-04-23T18:36:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.765+0000 7f8fc5e22740 20 client.15732 trim_cache size 0 max 16384 2024-04-23T18:36:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-04-23T18:36:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-04-23T18:36:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-04-23T18:36:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-04-23T18:36:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.766+0000 7f8fc310f640 1 -- 172.21.15.132:0/1373778139 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8fac0a3fe0 msgr2=0x7f8fac0a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.766+0000 7f8fc310f640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8fac0a3fe0 0x7f8fac0a64a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8fb402f540 tx=0x7f8fb4005df0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.766+0000 7f8fc310f640 1 -- 172.21.15.132:0/1373778139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fbc1010c0 msgr2=0x7f8fbc1975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.766+0000 7f8fc310f640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fbc1010c0 0x7f8fbc1975f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000e9f0 tx=0x7f8fb000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.766+0000 7f8fc310f640 1 -- 172.21.15.132:0/1373778139 shutdown_connections 2024-04-23T18:36:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.766+0000 7f8fc310f640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8fac0a3fe0 0x7f8fac0a64a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.766+0000 7f8fc310f640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fbc19beb0 0x7f8fbc19c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.766+0000 7f8fc310f640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fbc106190 0x7f8fbc197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.766+0000 7f8fc310f640 1 --2- 172.21.15.132:0/1373778139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fbc1010c0 0x7f8fbc1975f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.766+0000 7f8fc310f640 1 -- 172.21.15.132:0/1373778139 >> 172.21.15.132:0/1373778139 conn(0x7f8fbc077f70 msgr2=0x7f8fbc0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.766+0000 7f8fc310f640 1 -- 172.21.15.132:0/1373778139 shutdown_connections 2024-04-23T18:36:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:31.766+0000 7f8fc310f640 1 -- 172.21.15.132:0/1373778139 wait complete. 2024-04-23T18:36:32.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:32 smithi132 ceph-mon[29178]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-23T18:36:33.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:32 smithi148 ceph-mon[32311]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-23T18:36:33.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:32 smithi157 ceph-mon[32580]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-23T18:36:34.842 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:34 smithi157 ceph-mon[32580]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:36:34.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:34 smithi132 ceph-mon[29178]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:36:35.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:34 smithi148 ceph-mon[32311]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:36:36.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:36 smithi132 ceph-mon[29178]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-23T18:36:37.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:36 smithi148 ceph-mon[32311]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-23T18:36:37.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:36 smithi157 ceph-mon[32580]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-23T18:36:38.588 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:38 smithi132 ceph-mon[29178]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T18:36:39.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:38 smithi148 ceph-mon[32311]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T18:36:39.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:38 smithi157 ceph-mon[32580]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T18:36:40.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:40 smithi132 ceph-mon[29178]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-23T18:36:41.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:40 smithi148 ceph-mon[32311]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-23T18:36:41.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:40 smithi157 ceph-mon[32580]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-23T18:36:42.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:42 smithi132 ceph-mon[29178]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-23T18:36:43.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:42 smithi148 ceph-mon[32311]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-23T18:36:43.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:42 smithi157 ceph-mon[32580]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-23T18:36:44.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:44 smithi132 ceph-mon[29178]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-23T18:36:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.901+0000 7f47f6a91640 1 -- 172.21.15.132:0/2384089234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f0101aa0 msgr2=0x7f47f0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.901+0000 7f47f6a91640 1 --2- 172.21.15.132:0/2384089234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f0101aa0 0x7f47f0105a90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009900 tx=0x7f47dc02a410 comp rx=0 tx=0).stop 2024-04-23T18:36:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.901+0000 7f47f6a91640 1 -- 172.21.15.132:0/2384089234 shutdown_connections 2024-04-23T18:36:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.901+0000 7f47f6a91640 1 --2- 172.21.15.132:0/2384089234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47f0106170 0x7f47f010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.901+0000 7f47f6a91640 1 --2- 172.21.15.132:0/2384089234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f0101aa0 0x7f47f0105a90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.901+0000 7f47f6a91640 1 --2- 172.21.15.132:0/2384089234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f01010f0 0x7f47f01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.901+0000 7f47f6a91640 1 -- 172.21.15.132:0/2384089234 >> 172.21.15.132:0/2384089234 conn(0x7f47f0077fd0 msgr2=0x7f47f00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.901+0000 7f47f6a91640 1 -- 172.21.15.132:0/2384089234 shutdown_connections 2024-04-23T18:36:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.901+0000 7f47f6a91640 1 -- 172.21.15.132:0/2384089234 wait complete. 2024-04-23T18:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.901+0000 7f47f6a91640 1 Processor -- start 2024-04-23T18:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.902+0000 7f47f6a91640 1 -- start start 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.902+0000 7f47f6a91640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f01010f0 0x7f47f01994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.902+0000 7f47f6a91640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f0106170 0x7f47f0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.902+0000 7f47f6a91640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47f0199f40 0x7f47f01a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.902+0000 7f47effff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f0106170 0x7f47f0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.902+0000 7f47effff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f0106170 0x7f47f0199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46680/0 (socket says 172.21.15.132:46680) 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.902+0000 7f47effff640 1 -- 172.21.15.132:0/3144345846 learned_addr learned my addr 172.21.15.132:0/3144345846 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.902+0000 7f47f6a91640 1 -- 172.21.15.132:0/3144345846 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01110e0 con 0x7f47f0106170 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.902+0000 7f47f6a91640 1 -- 172.21.15.132:0/3144345846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0110f60 con 0x7f47f01010f0 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.902+0000 7f47f6a91640 1 -- 172.21.15.132:0/3144345846 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0111260 con 0x7f47f0199f40 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.902+0000 7f47f482d640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f01010f0 0x7f47f01994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.902+0000 7f47f502e640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47f0199f40 0x7f47f01a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.903+0000 7f47effff640 1 -- 172.21.15.132:0/3144345846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47f0199f40 msgr2=0x7f47f01a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.903+0000 7f47effff640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47f0199f40 0x7f47f01a4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.903+0000 7f47effff640 1 -- 172.21.15.132:0/3144345846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f01010f0 msgr2=0x7f47f01994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.903+0000 7f47effff640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f01010f0 0x7f47f01994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.903+0000 7f47effff640 1 -- 172.21.15.132:0/3144345846 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f01a4ce0 con 0x7f47f0106170 2024-04-23T18:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.903+0000 7f47f482d640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f01010f0 0x7f47f01994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.903+0000 7f47effff640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f0106170 0x7f47f0199a00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f47dc005ab0 tx=0x7f47dc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.903+0000 7f47edffb640 1 -- 172.21.15.132:0/3144345846 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc03f070 con 0x7f47f0106170 2024-04-23T18:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.903+0000 7f47f6a91640 1 -- 172.21.15.132:0/3144345846 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f01a4f70 con 0x7f47f0106170 2024-04-23T18:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.903+0000 7f47edffb640 1 -- 172.21.15.132:0/3144345846 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc031b50 con 0x7f47f0106170 2024-04-23T18:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.903+0000 7f47edffb640 1 -- 172.21.15.132:0/3144345846 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc035030 con 0x7f47f0106170 2024-04-23T18:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.903+0000 7f47f6a91640 1 -- 172.21.15.132:0/3144345846 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f01a5430 con 0x7f47f0106170 2024-04-23T18:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.907+0000 7f47edffb640 1 -- 172.21.15.132:0/3144345846 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f47dc042070 con 0x7f47f0106170 2024-04-23T18:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.907+0000 7f47f97ab740 1 Processor -- start 2024-04-23T18:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.907+0000 7f47f97ab740 1 -- start start 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.907+0000 7f47f97ab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e4f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.907+0000 7f47f97ab740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 0x56010e5042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.907+0000 7f47f97ab740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e504800 0x56010e506bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.907+0000 7f47f97ab740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56010e45e970 con 0x56010e4f9f40 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.907+0000 7f47f97ab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56010e3fb4a0 con 0x56010e4f9620 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.907+0000 7f47f97ab740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56010e45a930 con 0x56010e504800 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.907+0000 7f47f482d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 0x56010e5042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.907+0000 7f47f502e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e4f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.907+0000 7f47f482d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 0x56010e5042c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46696/0 (socket says 172.21.15.132:46696) 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47f482d640 1 -- 172.21.15.132:0/845047438 learned_addr learned my addr 172.21.15.132:0/845047438 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47edffb640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f47e00a4120 0x7f47e00a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47f482d640 1 -- 172.21.15.132:0/845047438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e504800 msgr2=0x56010e506bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47effff640 1 --2- 172.21.15.132:0/845047438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e504800 0x56010e506bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47f502e640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f47e00a4120 0x7f47e00a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47f482d640 1 --2- 172.21.15.132:0/845047438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e504800 0x56010e506bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47f482d640 1 -- 172.21.15.132:0/845047438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 msgr2=0x56010e4f9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47f482d640 1 --2- 172.21.15.132:0/845047438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e4f9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47edffb640 1 -- 172.21.15.132:0/3144345846 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f47dc03ba10 con 0x7f47f0106170 2024-04-23T18:36:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47f482d640 1 -- 172.21.15.132:0/845047438 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56010e171210 con 0x56010e4f9f40 2024-04-23T18:36:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47effff640 1 --2- 172.21.15.132:0/845047438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e504800 0x56010e506bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47f482d640 1 --2- 172.21.15.132:0/845047438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 0x56010e5042c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f47e4011640 tx=0x7f47e4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce08da76165cebf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.908+0000 7f47f502e640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f47e00a4120 0x7f47e00a65e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f47e8013920 tx=0x7f47e8004ab0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.909+0000 7f47daffd640 1 -- 172.21.15.132:0/845047438 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e401e070 con 0x56010e4f9f40 2024-04-23T18:36:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.909+0000 7f47daffd640 1 -- 172.21.15.132:0/845047438 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47e400fb10 con 0x56010e4f9f40 2024-04-23T18:36:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.909+0000 7f47daffd640 1 -- 172.21.15.132:0/845047438 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4019c30 con 0x56010e4f9f40 2024-04-23T18:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.910+0000 7f47f97ab740 1 -- 172.21.15.132:0/845047438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 msgr2=0x56010e5042c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.910+0000 7f47f97ab740 1 --2- 172.21.15.132:0/845047438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 0x56010e5042c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f47e4011640 tx=0x7f47e4011b00 comp rx=0 tx=0).stop 2024-04-23T18:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.910+0000 7f47f97ab740 1 -- 172.21.15.132:0/845047438 shutdown_connections 2024-04-23T18:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.910+0000 7f47f97ab740 1 --2- 172.21.15.132:0/845047438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e504800 0x56010e506bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.910+0000 7f47f97ab740 1 --2- 172.21.15.132:0/845047438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 0x56010e5042c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.910+0000 7f47f97ab740 1 --2- 172.21.15.132:0/845047438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e4f9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.910+0000 7f47f97ab740 1 -- 172.21.15.132:0/845047438 >> 172.21.15.132:0/845047438 conn(0x56010e4f5b30 msgr2=0x56010e4f5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.910+0000 7f47f97ab740 1 -- 172.21.15.132:0/845047438 shutdown_connections 2024-04-23T18:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.910+0000 7f47f97ab740 1 -- 172.21.15.132:0/845047438 wait complete. 2024-04-23T18:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.910+0000 7f47f97ab740 1 -- start start 2024-04-23T18:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.910+0000 7f47f97ab740 1 Processor -- start 2024-04-23T18:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.911+0000 7f47f97ab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e504160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.911+0000 7f47f97ab740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e4f9f40 0x56010e501450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.911+0000 7f47f97ab740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 0x56010e5019b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.911+0000 7f47f97ab740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56010e507430 con 0x56010e504800 2024-04-23T18:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.911+0000 7f47f97ab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56010e3f90b0 con 0x56010e4f9620 2024-04-23T18:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.911+0000 7f47f97ab740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56010e5075b0 con 0x56010e4f9f40 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.911+0000 7f47f482d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e504160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.911+0000 7f47f482d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e504160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.132:50958/0 (socket says 172.21.15.132:50958) 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.911+0000 7f47f482d640 1 -- 172.21.15.132:0/3927506920 learned_addr learned my addr 172.21.15.132:0/3927506920 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.911+0000 7f47f502e640 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 0x56010e5019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47effff640 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e4f9f40 0x56010e501450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47f97ab740 10 client.0 _ll_register_callbacks cb 0x56010e3e6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47f482d640 1 -- 172.21.15.132:0/3927506920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e4f9f40 msgr2=0x56010e501450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47f482d640 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e4f9f40 0x56010e501450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47f482d640 1 -- 172.21.15.132:0/3927506920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 msgr2=0x56010e5019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47f482d640 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 0x56010e5019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47f482d640 1 -- 172.21.15.132:0/3927506920 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56010e23d620 con 0x56010e4f9620 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47effff640 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e4f9f40 0x56010e501450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47f502e640 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 0x56010e5019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47f97ab740 1 -- 172.21.15.132:0/3927506920 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e400fb10 con 0x56010e4f9620 2024-04-23T18:36:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47f482d640 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e504160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f47e400a010 tx=0x7f47e400fef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80f666725da842c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47cffff640 10 client.25250 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47cffff640 1 -- 172.21.15.132:0/3927506920 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4027060 con 0x56010e4f9620 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.912+0000 7f47cffff640 1 -- 172.21.15.132:0/3927506920 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f47e400f660 con 0x56010e4f9620 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.913+0000 7f47cffff640 1 -- 172.21.15.132:0/3927506920 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47e401e040 con 0x56010e4f9620 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.913+0000 7f47f97ab740 10 client.25250 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.913+0000 7f47f97ab740 1 -- 172.21.15.132:0/3927506920 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({fsmap=26}) -- 0x56010e5131f0 con 0x56010e4f9620 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.913+0000 7f47cffff640 1 -- 172.21.15.132:0/3927506920 <== mon.2 v2:172.21.15.148:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f47e4021030 con 0x56010e4f9620 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.913+0000 7f47f97ab740 10 client.25250 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.913+0000 7f47f97ab740 10 client.25250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.913+0000 7f47f97ab740 20 client.25250 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.913+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56010e514540 0x56010e516930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.913+0000 7f47f97ab740 4 client.25250 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.913+0000 7f47f97ab740 1 -- 172.21.15.132:0/3927506920 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56010e46c390 con 0x56010e514540 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.913+0000 7f47effff640 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56010e514540 0x56010e516930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.914+0000 7f47cffff640 10 client.25250 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.914+0000 7f47effff640 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56010e514540 0x56010e516930 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c90e0eddcfaf033f server_cookie=24716a66123b9c2a in_seq=0 out_seq=0 2024-04-23T18:36:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47cffff640 1 -- 172.21.15.132:0/3927506920 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56010e46c390 con 0x56010e514540 2024-04-23T18:36:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47cffff640 10 client.25250 handle_command_reply: tid=0 2024-04-23T18:36:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47f97ab740 1 client.25250 shutdown 2024-04-23T18:36:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47f97ab740 1 -- 172.21.15.132:0/3927506920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 msgr2=0x56010e504160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e504160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f47e400a010 tx=0x7f47e400fef0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47f97ab740 1 -- 172.21.15.132:0/3927506920 shutdown_connections 2024-04-23T18:36:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56010e514540 0x56010e516930 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e4f9f40 0x56010e501450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47cffff640 0 client.25250 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 0x56010e5019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3927506920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e504160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47f97ab740 1 -- 172.21.15.132:0/3927506920 >> 172.21.15.132:0/3927506920 conn(0x56010e4f5b30 msgr2=0x56010e4fb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47f97ab740 1 -- 172.21.15.132:0/3927506920 shutdown_connections 2024-04-23T18:36:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.915+0000 7f47f97ab740 1 -- 172.21.15.132:0/3927506920 wait complete. 2024-04-23T18:36:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.916+0000 7f47f97ab740 20 client.25250 trim_cache size 0 max 16384 2024-04-23T18:36:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.944+0000 7f47f97ab740 1 Processor -- start 2024-04-23T18:36:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.944+0000 7f47f97ab740 1 -- start start 2024-04-23T18:36:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.944+0000 7f47f97ab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e503c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.944+0000 7f47f97ab740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e4f9f40 0x56010e51c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.944+0000 7f47f97ab740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 0x56010e51c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.944+0000 7f47f97ab740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01ab890 con 0x56010e504800 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.944+0000 7f47f97ab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01ad3d0 con 0x56010e4f9620 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.944+0000 7f47f97ab740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01abf60 con 0x56010e4f9f40 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47f502e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 0x56010e51c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47f502e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 0x56010e51c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46710/0 (socket says 172.21.15.132:46710) 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47f502e640 1 -- 172.21.15.132:0/3575517363 learned_addr learned my addr 172.21.15.132:0/3575517363 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47effff640 1 --2- 172.21.15.132:0/3575517363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e4f9f40 0x56010e51c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47f482d640 1 --2- 172.21.15.132:0/3575517363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e503c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47f502e640 1 -- 172.21.15.132:0/3575517363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e4f9f40 msgr2=0x56010e51c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47f502e640 1 --2- 172.21.15.132:0/3575517363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e4f9f40 0x56010e51c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47f502e640 1 -- 172.21.15.132:0/3575517363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 msgr2=0x56010e503c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47f502e640 1 --2- 172.21.15.132:0/3575517363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e503c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47f502e640 1 -- 172.21.15.132:0/3575517363 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56010e4fa8e0 con 0x56010e504800 2024-04-23T18:36:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47effff640 1 --2- 172.21.15.132:0/3575517363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e4f9f40 0x56010e51c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47f502e640 1 --2- 172.21.15.132:0/3575517363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 0x56010e51c580 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f47e8007960 tx=0x7f47e8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a52c8707881ebdc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.945+0000 7f47da7fc640 1 -- 172.21.15.132:0/3575517363 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8019040 con 0x56010e504800 2024-04-23T18:36:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.946+0000 7f47da7fc640 1 -- 172.21.15.132:0/3575517363 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47e801a040 con 0x56010e504800 2024-04-23T18:36:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.946+0000 7f47da7fc640 1 -- 172.21.15.132:0/3575517363 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e80046b0 con 0x56010e504800 2024-04-23T18:36:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.946+0000 7f47f97ab740 1 -- 172.21.15.132:0/3575517363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 msgr2=0x56010e51c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.946+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3575517363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 0x56010e51c580 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f47e8007960 tx=0x7f47e8007a40 comp rx=0 tx=0).stop 2024-04-23T18:36:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.947+0000 7f47f97ab740 1 -- 172.21.15.132:0/3575517363 shutdown_connections 2024-04-23T18:36:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.947+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3575517363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e4f9f40 0x56010e51c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.947+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3575517363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e504800 0x56010e51c580 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.947+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3575517363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e503c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.947+0000 7f47f97ab740 1 -- 172.21.15.132:0/3575517363 >> 172.21.15.132:0/3575517363 conn(0x56010e4f5b30 msgr2=0x56010e501240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.947+0000 7f47f97ab740 1 -- 172.21.15.132:0/3575517363 shutdown_connections 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.947+0000 7f47f97ab740 1 -- 172.21.15.132:0/3575517363 wait complete. 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.947+0000 7f47f97ab740 1 -- start start 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.947+0000 7f47f97ab740 1 Processor -- start 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f97ab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e5099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f97ab740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 0x56010e509f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f97ab740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e504800 0x56010e5788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f97ab740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01acd00 con 0x56010e4f9f40 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f97ab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01ac630 con 0x56010e4f9620 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f97ab740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01a9870 con 0x56010e504800 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f97ab740 10 client.0 _ll_register_callbacks cb 0x7f47e4011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f482d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e5099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f482d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e5099c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.132:50984/0 (socket says 172.21.15.132:50984) 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f482d640 1 -- 172.21.15.132:0/3756959815 learned_addr learned my addr 172.21.15.132:0/3756959815 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47effff640 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 0x56010e509f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f502e640 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e504800 0x56010e5788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f482d640 1 -- 172.21.15.132:0/3756959815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e504800 msgr2=0x56010e5788b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f482d640 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e504800 0x56010e5788b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f482d640 1 -- 172.21.15.132:0/3756959815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 msgr2=0x56010e509f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f482d640 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 0x56010e509f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f482d640 1 -- 172.21.15.132:0/3756959815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56010e578df0 con 0x56010e4f9620 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47effff640 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 0x56010e509f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.948+0000 7f47f482d640 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e5099c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f47e401abc0 tx=0x7f47e4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c30b10f90b0ca62 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47cffff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47cffff640 1 -- 172.21.15.132:0/3756959815 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4003800 con 0x56010e4f9620 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47f97ab740 1 -- 172.21.15.132:0/3756959815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e801a040 con 0x56010e4f9620 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47cffff640 1 -- 172.21.15.132:0/3756959815 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f47e40039a0 con 0x56010e4f9620 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47cffff640 1 -- 172.21.15.132:0/3756959815 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47e4003ba0 con 0x56010e4f9620 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47f97ab740 10 client.25256 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47f97ab740 1 -- 172.21.15.132:0/3756959815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56010e582910 con 0x56010e4f9620 2024-04-23T18:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47cffff640 1 -- 172.21.15.132:0/3756959815 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47e801a040 con 0x56010e4f9620 2024-04-23T18:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47cffff640 1 -- 172.21.15.132:0/3756959815 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f47e4033070 con 0x56010e4f9620 2024-04-23T18:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47f97ab740 10 client.25256 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47f97ab740 10 client.25256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47f97ab740 20 client.25256 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56010e514540 0x56010e584fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47f97ab740 4 client.25256 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.949+0000 7f47f97ab740 1 -- 172.21.15.132:0/3756959815 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56010e466200 con 0x56010e514540 2024-04-23T18:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.950+0000 7f47effff640 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56010e514540 0x56010e584fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.950+0000 7f47cffff640 10 client.25256 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.950+0000 7f47effff640 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56010e514540 0x56010e584fd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56fd5981ca0ac568 server_cookie=e2cac071cf197013 in_seq=0 out_seq=0 2024-04-23T18:36:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47cffff640 1 -- 172.21.15.132:0/3756959815 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56010e466200 con 0x56010e514540 2024-04-23T18:36:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47cffff640 10 client.25256 handle_command_reply: tid=0 2024-04-23T18:36:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47f97ab740 1 client.25256 shutdown 2024-04-23T18:36:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47f97ab740 1 -- 172.21.15.132:0/3756959815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 msgr2=0x56010e5099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e5099c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f47e401abc0 tx=0x7f47e4005f90 comp rx=0 tx=0).stop 2024-04-23T18:36:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47f97ab740 1 -- 172.21.15.132:0/3756959815 shutdown_connections 2024-04-23T18:36:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x56010e514540 0x56010e584fd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47cffff640 0 client.25256 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56010e504800 0x56010e5788b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56010e4f9f40 0x56010e509f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47f97ab740 1 --2- 172.21.15.132:0/3756959815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56010e4f9620 0x56010e5099c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47f97ab740 1 -- 172.21.15.132:0/3756959815 >> 172.21.15.132:0/3756959815 conn(0x56010e4f5b30 msgr2=0x56010e4f4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47f97ab740 1 -- 172.21.15.132:0/3756959815 shutdown_connections 2024-04-23T18:36:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47f97ab740 1 -- 172.21.15.132:0/3756959815 wait complete. 2024-04-23T18:36:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.954+0000 7f47f97ab740 20 client.25256 trim_cache size 0 max 16384 2024-04-23T18:36:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-04-23T18:36:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-04-23T18:36:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-04-23T18:36:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-04-23T18:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.955+0000 7f47f6a91640 1 -- 172.21.15.132:0/3144345846 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f47e00a4120 msgr2=0x7f47e00a65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.955+0000 7f47f6a91640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f47e00a4120 0x7f47e00a65e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f47e8013920 tx=0x7f47e8004ab0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.955+0000 7f47f6a91640 1 -- 172.21.15.132:0/3144345846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f0106170 msgr2=0x7f47f0199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.955+0000 7f47f6a91640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f0106170 0x7f47f0199a00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f47dc005ab0 tx=0x7f47dc003b90 comp rx=0 tx=0).stop 2024-04-23T18:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.955+0000 7f47f6a91640 1 -- 172.21.15.132:0/3144345846 shutdown_connections 2024-04-23T18:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.955+0000 7f47f6a91640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f47e00a4120 0x7f47e00a65e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.955+0000 7f47f6a91640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47f0199f40 0x7f47f01a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.955+0000 7f47f6a91640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47f0106170 0x7f47f0199a00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.955+0000 7f47f6a91640 1 --2- 172.21.15.132:0/3144345846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f01010f0 0x7f47f01994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.955+0000 7f47f6a91640 1 -- 172.21.15.132:0/3144345846 >> 172.21.15.132:0/3144345846 conn(0x7f47f0077fd0 msgr2=0x7f47f00feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.955+0000 7f47f6a91640 1 -- 172.21.15.132:0/3144345846 shutdown_connections 2024-04-23T18:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:44.955+0000 7f47f6a91640 1 -- 172.21.15.132:0/3144345846 wait complete. 2024-04-23T18:36:45.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:44 smithi148 ceph-mon[32311]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-23T18:36:45.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:44 smithi157 ceph-mon[32580]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-23T18:36:46.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:46 smithi132 ceph-mon[29178]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-23T18:36:47.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:46 smithi148 ceph-mon[32311]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-23T18:36:47.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:46 smithi157 ceph-mon[32580]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-23T18:36:47.381 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:36:47.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.491+0000 7f5972864640 1 -- 172.21.15.132:0/4176587852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596c103b00 msgr2=0x7f596c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:47.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.491+0000 7f5972864640 1 --2- 172.21.15.132:0/4176587852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596c103b00 0x7f596c10bfe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5960009900 tx=0x7f596002a510 comp rx=0 tx=0).stop 2024-04-23T18:36:47.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.491+0000 7f5972864640 1 -- 172.21.15.132:0/4176587852 shutdown_connections 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.491+0000 7f5972864640 1 --2- 172.21.15.132:0/4176587852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596c103b00 0x7f596c10bfe0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.491+0000 7f5972864640 1 --2- 172.21.15.132:0/4176587852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f596c10c520 0x7f596c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.491+0000 7f5972864640 1 --2- 172.21.15.132:0/4176587852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f596c1031e0 0x7f596c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.491+0000 7f5972864640 1 -- 172.21.15.132:0/4176587852 >> 172.21.15.132:0/4176587852 conn(0x7f596c077f50 msgr2=0x7f596c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.491+0000 7f5972864640 1 -- 172.21.15.132:0/4176587852 shutdown_connections 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.491+0000 7f5972864640 1 -- 172.21.15.132:0/4176587852 wait complete. 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.491+0000 7f5972864640 1 Processor -- start 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.492+0000 7f5972864640 1 -- start start 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.492+0000 7f5972864640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596c1031e0 0x7f596c19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.492+0000 7f5972864640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f596c103b00 0x7f596c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.492+0000 7f5972864640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f596c10c520 0x7f596c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.492+0000 7f5972864640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1121b0 con 0x7f596c103b00 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.492+0000 7f5972864640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f596c112030 con 0x7f596c10c520 2024-04-23T18:36:47.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.492+0000 7f5972864640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f596c112330 con 0x7f596c1031e0 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.492+0000 7f596b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f596c103b00 0x7f596c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.492+0000 7f596b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f596c103b00 0x7f596c19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46748/0 (socket says 172.21.15.132:46748) 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.492+0000 7f596b7fe640 1 -- 172.21.15.132:0/686133450 learned_addr learned my addr 172.21.15.132:0/686133450 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.492+0000 7f596bfff640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596c1031e0 0x7f596c19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.492+0000 7f5970e01640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f596c10c520 0x7f596c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f596b7fe640 1 -- 172.21.15.132:0/686133450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596c1031e0 msgr2=0x7f596c19e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f596b7fe640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596c1031e0 0x7f596c19e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f596b7fe640 1 -- 172.21.15.132:0/686133450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f596c10c520 msgr2=0x7f596c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f596b7fe640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f596c10c520 0x7f596c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f596b7fe640 1 -- 172.21.15.132:0/686133450 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c198d30 con 0x7f596c103b00 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f5970e01640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f596c10c520 0x7f596c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f596bfff640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596c1031e0 0x7f596c19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f596b7fe640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f596c103b00 0x7f596c19e940 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5960005940 tx=0x7f59600040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f59697fa640 1 -- 172.21.15.132:0/686133450 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596003f030 con 0x7f596c103b00 2024-04-23T18:36:47.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f5972864640 1 -- 172.21.15.132:0/686133450 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c198fc0 con 0x7f596c103b00 2024-04-23T18:36:47.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f59697fa640 1 -- 172.21.15.132:0/686133450 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59600044c0 con 0x7f596c103b00 2024-04-23T18:36:47.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f5972864640 1 -- 172.21.15.132:0/686133450 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c100d60 con 0x7f596c103b00 2024-04-23T18:36:47.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.493+0000 7f59697fa640 1 -- 172.21.15.132:0/686133450 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960035070 con 0x7f596c103b00 2024-04-23T18:36:47.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.494+0000 7f5972864640 1 -- 172.21.15.132:0/686133450 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596c068d70 con 0x7f596c103b00 2024-04-23T18:36:47.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.495+0000 7f59697fa640 1 -- 172.21.15.132:0/686133450 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f5960042030 con 0x7f596c103b00 2024-04-23T18:36:47.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.496+0000 7f59697fa640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f59540a3e40 0x7f59540a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:47.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.496+0000 7f596bfff640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f59540a3e40 0x7f59540a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:47.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.496+0000 7f59697fa640 1 -- 172.21.15.132:0/686133450 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f59600f0dc0 con 0x7f596c103b00 2024-04-23T18:36:47.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.497+0000 7f596bfff640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f59540a3e40 0x7f59540a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f595c009fc0 tx=0x7f595c005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:47.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.504+0000 7f59697fa640 1 -- 172.21.15.132:0/686133450 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59600bd2d0 con 0x7f596c103b00 2024-04-23T18:36:47.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.848+0000 7f5972864640 1 -- 172.21.15.132:0/686133450 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f596c1076a0 con 0x7f596c103b00 2024-04-23T18:36:47.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.849+0000 7f59697fa640 1 -- 172.21.15.132:0/686133450 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11946 (secure 0 0 0) 0x7f59600bcbd0 con 0x7f596c103b00 2024-04-23T18:36:47.841 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:47.842 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:36:01.455959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24460,24425,24394,24491,24515]},"id":1}]} 2024-04-23T18:36:47.842 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 26 2024-04-23T18:36:47.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.852+0000 7f5972864640 1 -- 172.21.15.132:0/686133450 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f59540a3e40 msgr2=0x7f59540a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:47.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.852+0000 7f5972864640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f59540a3e40 0x7f59540a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f595c009fc0 tx=0x7f595c005f90 comp rx=0 tx=0).stop 2024-04-23T18:36:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.852+0000 7f5972864640 1 -- 172.21.15.132:0/686133450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f596c103b00 msgr2=0x7f596c19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.852+0000 7f5972864640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f596c103b00 0x7f596c19e940 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5960005940 tx=0x7f59600040c0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.853+0000 7f5972864640 1 -- 172.21.15.132:0/686133450 shutdown_connections 2024-04-23T18:36:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.853+0000 7f5972864640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f59540a3e40 0x7f59540a6300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.853+0000 7f5972864640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f596c10c520 0x7f596c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.853+0000 7f5972864640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f596c103b00 0x7f596c19e940 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.853+0000 7f5972864640 1 --2- 172.21.15.132:0/686133450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596c1031e0 0x7f596c19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.853+0000 7f5972864640 1 -- 172.21.15.132:0/686133450 >> 172.21.15.132:0/686133450 conn(0x7f596c077f50 msgr2=0x7f596c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.853+0000 7f5972864640 1 -- 172.21.15.132:0/686133450 shutdown_connections 2024-04-23T18:36:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:47.853+0000 7f5972864640 1 -- 172.21.15.132:0/686133450 wait complete. 2024-04-23T18:36:47.856 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24515 quiesce db --set-id a559ed --release --await' 2024-04-23T18:36:47.856 DEBUG:teuthology.orchestra.run.smithi132:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24515 quiesce db --set-id a559ed --release --await 2024-04-23T18:36:47.997 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.005+0000 7f81bcedc640 1 -- 172.21.15.132:0/3441008374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b8102b50 msgr2=0x7f81b8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:47.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.005+0000 7f81bcedc640 1 --2- 172.21.15.132:0/3441008374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b8102b50 0x7f81b8106b90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009900 tx=0x7f81ac02a410 comp rx=0 tx=0).stop 2024-04-23T18:36:47.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.005+0000 7f81bcedc640 1 -- 172.21.15.132:0/3441008374 shutdown_connections 2024-04-23T18:36:47.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.005+0000 7f81bcedc640 1 --2- 172.21.15.132:0/3441008374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b8107270 0x7f81b810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.005+0000 7f81bcedc640 1 --2- 172.21.15.132:0/3441008374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b8102b50 0x7f81b8106b90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.005+0000 7f81bcedc640 1 --2- 172.21.15.132:0/3441008374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b81021a0 0x7f81b8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:47.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.005+0000 7f81bcedc640 1 -- 172.21.15.132:0/3441008374 >> 172.21.15.132:0/3441008374 conn(0x7f81b8079350 msgr2=0x7f81b8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:47.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.005+0000 7f81bcedc640 1 -- 172.21.15.132:0/3441008374 shutdown_connections 2024-04-23T18:36:47.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.005+0000 7f81bcedc640 1 -- 172.21.15.132:0/3441008374 wait complete. 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.006+0000 7f81bcedc640 1 Processor -- start 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.006+0000 7f81bcedc640 1 -- start start 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81bcedc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b81021a0 0x7f81b819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81bcedc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b8107270 0x7f81b819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81bcedc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b8198600 0x7f81b8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81bcedc640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81121b0 con 0x7f81b8107270 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81bcedc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f81b8112030 con 0x7f81b8198600 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81bcedc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81b8112330 con 0x7f81b81021a0 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b8107270 0x7f81b819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b8107270 0x7f81b819e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46758/0 (socket says 172.21.15.132:46758) 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b5d9b640 1 -- 172.21.15.132:0/144385683 learned_addr learned my addr 172.21.15.132:0/144385683 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b659c640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b81021a0 0x7f81b819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b6d9d640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b8198600 0x7f81b8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b5d9b640 1 -- 172.21.15.132:0/144385683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b81021a0 msgr2=0x7f81b819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b5d9b640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b81021a0 0x7f81b819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b5d9b640 1 -- 172.21.15.132:0/144385683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b8198600 msgr2=0x7f81b8198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b5d9b640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b8198600 0x7f81b8198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b5d9b640 1 -- 172.21.15.132:0/144385683 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b8199120 con 0x7f81b8107270 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b6d9d640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b8198600 0x7f81b8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b659c640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b81021a0 0x7f81b819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.007+0000 7f81b5d9b640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b8107270 0x7f81b819e940 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f81ac032fd0 tx=0x7f81ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.008+0000 7f81a77fe640 1 -- 172.21.15.132:0/144385683 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac03f030 con 0x7f81b8107270 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.008+0000 7f81bcedc640 1 -- 172.21.15.132:0/144385683 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b81993b0 con 0x7f81b8107270 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.008+0000 7f81a77fe640 1 -- 172.21.15.132:0/144385683 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ac004490 con 0x7f81b8107270 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.008+0000 7f81a77fe640 1 -- 172.21.15.132:0/144385683 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac035070 con 0x7f81b8107270 2024-04-23T18:36:48.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.008+0000 7f81bcedc640 1 -- 172.21.15.132:0/144385683 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b81a55b0 con 0x7f81b8107270 2024-04-23T18:36:48.004 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.010+0000 7f81a77fe640 1 -- 172.21.15.132:0/144385683 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f81ac042030 con 0x7f81b8107270 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.011+0000 7f81a77fe640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f81a00a3ec0 0x7f81a00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.011+0000 7f81b659c640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f81a00a3ec0 0x7f81a00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.011+0000 7f81a77fe640 1 -- 172.21.15.132:0/144385683 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f81ac0f0ee0 con 0x7f81b8107270 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.011+0000 7f81bfbf0740 1 Processor -- start 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.011+0000 7f81bfbf0740 1 -- start start 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81bfbf0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3861490 0x5586e3861870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81bfbf0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3861db0 0x5586e386bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81bfbf0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e386c480 0x5586e386e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81bfbf0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5586e37c66d0 con 0x5586e3861490 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81bfbf0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5586e37631a0 con 0x5586e386c480 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81bfbf0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5586e37c2690 con 0x5586e3861db0 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3861490 0x5586e3861870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3861490 0x5586e3861870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46770/0 (socket says 172.21.15.132:46770) 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b659c640 1 -- 172.21.15.132:0/1746330129 learned_addr learned my addr 172.21.15.132:0/1746330129 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b659c640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f81a00a3ec0 0x7f81a00a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f81a80058c0 tx=0x7f81a8005850 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b5d9b640 1 --2- 172.21.15.132:0/1746330129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3861db0 0x5586e386bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b6d9d640 1 --2- 172.21.15.132:0/1746330129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e386c480 0x5586e386e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b659c640 1 -- 172.21.15.132:0/1746330129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3861db0 msgr2=0x5586e386bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b659c640 1 --2- 172.21.15.132:0/1746330129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3861db0 0x5586e386bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b659c640 1 -- 172.21.15.132:0/1746330129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e386c480 msgr2=0x5586e386e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b659c640 1 --2- 172.21.15.132:0/1746330129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e386c480 0x5586e386e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b659c640 1 -- 172.21.15.132:0/1746330129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586e376d5b0 con 0x5586e3861490 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b5d9b640 1 --2- 172.21.15.132:0/1746330129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3861db0 0x5586e386bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81b659c640 1 --2- 172.21.15.132:0/1746330129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3861490 0x5586e3861870 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f81a8005ca0 tx=0x7f81a8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a55b48e78516d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.012+0000 7f81a4ff9640 1 -- 172.21.15.132:0/1746330129 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8022070 con 0x5586e3861490 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.013+0000 7f81a4ff9640 1 -- 172.21.15.132:0/1746330129 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a801eb00 con 0x5586e3861490 2024-04-23T18:36:48.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.013+0000 7f81a4ff9640 1 -- 172.21.15.132:0/1746330129 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a801d9d0 con 0x5586e3861490 2024-04-23T18:36:48.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.013+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1746330129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3861490 msgr2=0x5586e3861870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.013+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/1746330129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3861490 0x5586e3861870 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f81a8005ca0 tx=0x7f81a8012000 comp rx=0 tx=0).stop 2024-04-23T18:36:48.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.013+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1746330129 shutdown_connections 2024-04-23T18:36:48.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.013+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/1746330129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3861db0 0x5586e386bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.013+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/1746330129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e386c480 0x5586e386e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.013+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/1746330129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3861490 0x5586e3861870 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.013+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1746330129 >> 172.21.15.132:0/1746330129 conn(0x5586e385d7b0 msgr2=0x5586e385db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.013+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1746330129 shutdown_connections 2024-04-23T18:36:48.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1746330129 wait complete. 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81bfbf0740 1 -- start start 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81bfbf0740 1 Processor -- start 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81bfbf0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e3869420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81bfbf0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e386c480 0x5586e3869960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81bfbf0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3869ea0 0x5586e3878780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81bfbf0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5586e386f310 con 0x5586e3869ea0 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81bfbf0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5586e3760b20 con 0x5586e3861db0 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81bfbf0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5586e386f490 con 0x5586e386c480 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81b6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3869ea0 0x5586e3878780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81bfbf0740 10 client.0 _ll_register_callbacks cb 0x5586e3760530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81b6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3869ea0 0x5586e3878780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46780/0 (socket says 172.21.15.132:46780) 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81b6d9d640 1 -- 172.21.15.132:0/961770108 learned_addr learned my addr 172.21.15.132:0/961770108 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:48.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81b659c640 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e3869420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.014+0000 7f81b5d9b640 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e386c480 0x5586e3869960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f81b6d9d640 1 -- 172.21.15.132:0/961770108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e386c480 msgr2=0x5586e3869960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f81b6d9d640 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e386c480 0x5586e3869960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f81b6d9d640 1 -- 172.21.15.132:0/961770108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 msgr2=0x5586e3869420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f81b6d9d640 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e3869420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f81b6d9d640 1 -- 172.21.15.132:0/961770108 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586e34d8920 con 0x5586e3869ea0 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f81b6d9d640 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3869ea0 0x5586e3878780 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f81b0012720 tx=0x7f81b0012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a3360331d3ca7f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f819dffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f819dffb640 1 -- 172.21.15.132:0/961770108 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b001f060 con 0x5586e3869ea0 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f819dffb640 1 -- 172.21.15.132:0/961770108 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f81b0010d60 con 0x5586e3869ea0 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f81bfbf0740 1 -- 172.21.15.132:0/961770108 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81a801eb00 con 0x5586e3869ea0 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f819dffb640 1 -- 172.21.15.132:0/961770108 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f81b0023420 con 0x5586e3869ea0 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f81bfbf0740 10 client.15768 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f81bfbf0740 1 -- 172.21.15.132:0/961770108 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5586e35a4d30 con 0x5586e3869ea0 2024-04-23T18:36:48.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.015+0000 7f819dffb640 1 -- 172.21.15.132:0/961770108 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a801eb00 con 0x5586e3869ea0 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.016+0000 7f819dffb640 1 -- 172.21.15.132:0/961770108 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f81b0019030 con 0x5586e3869ea0 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.016+0000 7f81bfbf0740 10 client.15768 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.016+0000 7f81bfbf0740 10 client.15768 resolve_mds: validated gid 24515 aka daemon mds.b 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.016+0000 7f81bfbf0740 20 client.15768 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.016+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5586e38810f0 0x5586e3883540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.016+0000 7f81bfbf0740 4 client.15768 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.016+0000 7f81bfbf0740 1 -- 172.21.15.132:0/961770108 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586e3784500 con 0x5586e38810f0 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.016+0000 7f81b659c640 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5586e38810f0 0x5586e3883540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.016+0000 7f81b659c640 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5586e38810f0 0x5586e3883540 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeda1a86a6a13eec server_cookie=979b9954f27338a0 in_seq=0 out_seq=0 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.016+0000 7f819dffb640 10 client.15768 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.017+0000 7f819dffb640 1 -- 172.21.15.132:0/961770108 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586e3784500 con 0x5586e38810f0 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.017+0000 7f819dffb640 10 client.15768 handle_command_reply: tid=0 2024-04-23T18:36:48.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.017+0000 7f81bfbf0740 1 client.15768 shutdown 2024-04-23T18:36:48.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.017+0000 7f81bfbf0740 1 -- 172.21.15.132:0/961770108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3869ea0 msgr2=0x5586e3878780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.017+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3869ea0 0x5586e3878780 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f81b0012720 tx=0x7f81b0012be0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.017+0000 7f81bfbf0740 1 -- 172.21.15.132:0/961770108 shutdown_connections 2024-04-23T18:36:48.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.017+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5586e38810f0 0x5586e3883540 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.017+0000 7f819dffb640 0 client.15768 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:48.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.017+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e386c480 0x5586e3869960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.017+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e3869ea0 0x5586e3878780 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.017+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/961770108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e3869420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.017+0000 7f81bfbf0740 1 -- 172.21.15.132:0/961770108 >> 172.21.15.132:0/961770108 conn(0x5586e385d7b0 msgr2=0x5586e38635b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.018+0000 7f81bfbf0740 1 -- 172.21.15.132:0/961770108 shutdown_connections 2024-04-23T18:36:48.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.018+0000 7f81bfbf0740 1 -- 172.21.15.132:0/961770108 wait complete. 2024-04-23T18:36:48.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.018+0000 7f81bfbf0740 20 client.15768 trim_cache size 0 max 16384 2024-04-23T18:36:48.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81bfbf0740 1 Processor -- start 2024-04-23T18:36:48.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81bfbf0740 1 -- start start 2024-04-23T18:36:48.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81bfbf0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e385d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81bfbf0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3869ea0 0x5586e3888e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81bfbf0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 0x5586e3889350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81bfbf0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81abad0 con 0x5586e386c480 2024-04-23T18:36:48.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81bfbf0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81ad610 con 0x5586e3861db0 2024-04-23T18:36:48.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81bfbf0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81ac1a0 con 0x5586e3869ea0 2024-04-23T18:36:48.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81b6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 0x5586e3889350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81b6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 0x5586e3889350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46794/0 (socket says 172.21.15.132:46794) 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81b6d9d640 1 -- 172.21.15.132:0/3912030726 learned_addr learned my addr 172.21.15.132:0/3912030726 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81b659c640 1 --2- 172.21.15.132:0/3912030726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e385d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.051+0000 7f81b5d9b640 1 --2- 172.21.15.132:0/3912030726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3869ea0 0x5586e3888e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.052+0000 7f81b6d9d640 1 -- 172.21.15.132:0/3912030726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3869ea0 msgr2=0x5586e3888e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.052+0000 7f81b6d9d640 1 --2- 172.21.15.132:0/3912030726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3869ea0 0x5586e3888e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.052+0000 7f81b6d9d640 1 -- 172.21.15.132:0/3912030726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 msgr2=0x5586e385d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.052+0000 7f81b6d9d640 1 --2- 172.21.15.132:0/3912030726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e385d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.052+0000 7f81b6d9d640 1 -- 172.21.15.132:0/3912030726 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586e3861be0 con 0x5586e386c480 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.052+0000 7f81b659c640 1 --2- 172.21.15.132:0/3912030726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e385d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.052+0000 7f81b6d9d640 1 --2- 172.21.15.132:0/3912030726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 0x5586e3889350 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f81b0006010 tx=0x7f81b0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc01c741417df211 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.052+0000 7f819ffff640 1 -- 172.21.15.132:0/3912030726 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0010d60 con 0x5586e386c480 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.052+0000 7f819ffff640 1 -- 172.21.15.132:0/3912030726 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b000cb80 con 0x5586e386c480 2024-04-23T18:36:48.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.052+0000 7f819ffff640 1 -- 172.21.15.132:0/3912030726 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0023c00 con 0x5586e386c480 2024-04-23T18:36:48.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.053+0000 7f81bfbf0740 1 -- 172.21.15.132:0/3912030726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 msgr2=0x5586e3889350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.053+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/3912030726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 0x5586e3889350 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f81b0006010 tx=0x7f81b0009580 comp rx=0 tx=0).stop 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.053+0000 7f81bfbf0740 1 -- 172.21.15.132:0/3912030726 shutdown_connections 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.053+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/3912030726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3869ea0 0x5586e3888e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.053+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/3912030726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 0x5586e3889350 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.053+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/3912030726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e385d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.053+0000 7f81bfbf0740 1 -- 172.21.15.132:0/3912030726 >> 172.21.15.132:0/3912030726 conn(0x5586e385d7b0 msgr2=0x5586e386ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.053+0000 7f81bfbf0740 1 -- 172.21.15.132:0/3912030726 shutdown_connections 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.053+0000 7f81bfbf0740 1 -- 172.21.15.132:0/3912030726 wait complete. 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.053+0000 7f81bfbf0740 1 -- start start 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81bfbf0740 1 Processor -- start 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81bfbf0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e3866490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81bfbf0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3869ea0 0x5586e38669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81bfbf0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 0x5586e38e6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81bfbf0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81acf40 con 0x5586e386c480 2024-04-23T18:36:48.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81bfbf0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81ac870 con 0x5586e3861db0 2024-04-23T18:36:48.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81bfbf0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81a9ad0 con 0x5586e3869ea0 2024-04-23T18:36:48.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81bfbf0740 10 client.0 _ll_register_callbacks cb 0x7f81b0009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:48.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81b6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 0x5586e38e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81b6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 0x5586e38e6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46804/0 (socket says 172.21.15.132:46804) 2024-04-23T18:36:48.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81b6d9d640 1 -- 172.21.15.132:0/1813423681 learned_addr learned my addr 172.21.15.132:0/1813423681 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:48.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81b659c640 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e3866490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81b5d9b640 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3869ea0 0x5586e38669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81b6d9d640 1 -- 172.21.15.132:0/1813423681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3869ea0 msgr2=0x5586e38669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81b6d9d640 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3869ea0 0x5586e38669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81b6d9d640 1 -- 172.21.15.132:0/1813423681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 msgr2=0x5586e3866490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81b6d9d640 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e3866490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.054+0000 7f81b6d9d640 1 -- 172.21.15.132:0/1813423681 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586e38e6bc0 con 0x5586e386c480 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.055+0000 7f81b6d9d640 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 0x5586e38e6680 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f81b0005c40 tx=0x7f81b0023870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee07e1c0d6f992 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.055+0000 7f819dffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.055+0000 7f819dffb640 1 -- 172.21.15.132:0/1813423681 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b001f040 con 0x5586e386c480 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.055+0000 7f819dffb640 1 -- 172.21.15.132:0/1813423681 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f81b000c850 con 0x5586e386c480 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.055+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1813423681 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81b000cb80 con 0x5586e386c480 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.055+0000 7f819dffb640 1 -- 172.21.15.132:0/1813423681 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f81b0043050 con 0x5586e386c480 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.055+0000 7f81bfbf0740 10 client.15774 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.055+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1813423681 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5586e38e7d60 con 0x5586e386c480 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.055+0000 7f819dffb640 1 -- 172.21.15.132:0/1813423681 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b000cb80 con 0x5586e386c480 2024-04-23T18:36:48.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.055+0000 7f819dffb640 1 -- 172.21.15.132:0/1813423681 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f81b003e030 con 0x5586e386c480 2024-04-23T18:36:48.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.055+0000 7f81bfbf0740 10 client.15774 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:48.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.055+0000 7f81bfbf0740 10 client.15774 resolve_mds: validated gid 24515 aka daemon mds.b 2024-04-23T18:36:48.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.056+0000 7f81bfbf0740 20 client.15774 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:48.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.056+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5586e38810f0 0x5586e38ea3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.056+0000 7f81bfbf0740 4 client.15774 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a559ed", "release": true, "await": true}] 2024-04-23T18:36:48.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.056+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1813423681 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a559ed", "release": true, "await": true}) -- 0x5586e37d40f0 con 0x5586e38810f0 2024-04-23T18:36:48.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.056+0000 7f81b659c640 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5586e38810f0 0x5586e38ea3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.056+0000 7f819dffb640 10 client.15774 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:48.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.056+0000 7f81b659c640 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5586e38810f0 0x5586e38ea3a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb75a53b4397971 server_cookie=ebe323696acb807 in_seq=0 out_seq=0 2024-04-23T18:36:48.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.058+0000 7f819dffb640 1 -- 172.21.15.132:0/1813423681 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+586 (crc 0 0 0) 0x5586e37d40f0 con 0x5586e38810f0 2024-04-23T18:36:48.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.058+0000 7f819dffb640 10 client.15774 handle_command_reply: tid=0 2024-04-23T18:36:48.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.058+0000 7f81bfbf0740 1 client.15774 shutdown 2024-04-23T18:36:48.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.059+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1813423681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 msgr2=0x5586e38e6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.059+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 0x5586e38e6680 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f81b0005c40 tx=0x7f81b0023870 comp rx=0 tx=0).stop 2024-04-23T18:36:48.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.059+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1813423681 shutdown_connections 2024-04-23T18:36:48.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.059+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x5586e38810f0 0x5586e38ea3a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.059+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586e3869ea0 0x5586e38669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.059+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586e386c480 0x5586e38e6680 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.059+0000 7f819dffb640 0 client.15774 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:48.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.059+0000 7f81bfbf0740 1 --2- 172.21.15.132:0/1813423681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586e3861db0 0x5586e3866490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.059+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1813423681 >> 172.21.15.132:0/1813423681 conn(0x5586e385d7b0 msgr2=0x5586e386b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.059+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1813423681 shutdown_connections 2024-04-23T18:36:48.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.059+0000 7f81bfbf0740 1 -- 172.21.15.132:0/1813423681 wait complete. 2024-04-23T18:36:48.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.059+0000 7f81bfbf0740 20 client.15774 trim_cache size 0 max 16384 2024-04-23T18:36:48.055 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:36:48.055 INFO:teuthology.orchestra.run.smithi132.stdout: "epoch": 26, 2024-04-23T18:36:48.055 INFO:teuthology.orchestra.run.smithi132.stdout: "leader": 24515, 2024-04-23T18:36:48.055 INFO:teuthology.orchestra.run.smithi132.stdout: "set_version": 4, 2024-04-23T18:36:48.055 INFO:teuthology.orchestra.run.smithi132.stdout: "sets": { 2024-04-23T18:36:48.055 INFO:teuthology.orchestra.run.smithi132.stdout: "a559ed": { 2024-04-23T18:36:48.055 INFO:teuthology.orchestra.run.smithi132.stdout: "version": 4, 2024-04-23T18:36:48.055 INFO:teuthology.orchestra.run.smithi132.stdout: "age_ref": 0.0, 2024-04-23T18:36:48.055 INFO:teuthology.orchestra.run.smithi132.stdout: "state": { 2024-04-23T18:36:48.055 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RELEASED", 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: "age": 0.0 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: "timeout": 60.0, 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: "expiration": 79.5, 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: "members": { 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: "file:/": { 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: "excluded": false, 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: "state": { 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RELEASED", 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: "age": 0.0 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:48.056 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:48.057 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:48.057 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:48.057 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:36:48.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.060+0000 7f81bcedc640 1 -- 172.21.15.132:0/144385683 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f81a00a3ec0 msgr2=0x7f81a00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.060+0000 7f81bcedc640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f81a00a3ec0 0x7f81a00a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f81a80058c0 tx=0x7f81a8005850 comp rx=0 tx=0).stop 2024-04-23T18:36:48.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.060+0000 7f81bcedc640 1 -- 172.21.15.132:0/144385683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b8107270 msgr2=0x7f81b819e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.060+0000 7f81bcedc640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b8107270 0x7f81b819e940 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f81ac032fd0 tx=0x7f81ac004270 comp rx=0 tx=0).stop 2024-04-23T18:36:48.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.060+0000 7f81bcedc640 1 -- 172.21.15.132:0/144385683 shutdown_connections 2024-04-23T18:36:48.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.060+0000 7f81bcedc640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f81a00a3ec0 0x7f81a00a6380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.060+0000 7f81bcedc640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b8198600 0x7f81b8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.060+0000 7f81bcedc640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b8107270 0x7f81b819e940 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.060+0000 7f81bcedc640 1 --2- 172.21.15.132:0/144385683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81b81021a0 0x7f81b819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.060+0000 7f81bcedc640 1 -- 172.21.15.132:0/144385683 >> 172.21.15.132:0/144385683 conn(0x7f81b8079350 msgr2=0x7f81b8100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.060+0000 7f81bcedc640 1 -- 172.21.15.132:0/144385683 shutdown_connections 2024-04-23T18:36:48.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.060+0000 7f81bcedc640 1 -- 172.21.15.132:0/144385683 wait complete. 2024-04-23T18:36:48.066 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a559ed', seconds elapsed: 20.7 2024-04-23T18:36:48.066 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.45 2024-04-23T18:36:48.066 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:36:48.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.173+0000 7f5c1d15d640 1 -- 172.21.15.132:0/4125746914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c18101090 msgr2=0x7f5c18101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.173+0000 7f5c1d15d640 1 --2- 172.21.15.132:0/4125746914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c18101090 0x7f5c18101470 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5c08009900 tx=0x7f5c0802a410 comp rx=0 tx=0).stop 2024-04-23T18:36:48.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.173+0000 7f5c1d15d640 1 -- 172.21.15.132:0/4125746914 shutdown_connections 2024-04-23T18:36:48.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.173+0000 7f5c1d15d640 1 --2- 172.21.15.132:0/4125746914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c18106160 0x7f5c1810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.173+0000 7f5c1d15d640 1 --2- 172.21.15.132:0/4125746914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c18101a40 0x7f5c18105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.173+0000 7f5c1d15d640 1 --2- 172.21.15.132:0/4125746914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c18101090 0x7f5c18101470 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.173+0000 7f5c1d15d640 1 -- 172.21.15.132:0/4125746914 >> 172.21.15.132:0/4125746914 conn(0x7f5c180771d0 msgr2=0x7f5c180775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.173+0000 7f5c1d15d640 1 -- 172.21.15.132:0/4125746914 shutdown_connections 2024-04-23T18:36:48.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.173+0000 7f5c1d15d640 1 -- 172.21.15.132:0/4125746914 wait complete. 2024-04-23T18:36:48.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.174+0000 7f5c1d15d640 1 Processor -- start 2024-04-23T18:36:48.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.174+0000 7f5c1d15d640 1 -- start start 2024-04-23T18:36:48.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.174+0000 7f5c1d15d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c18101090 0x7f5c181995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.174+0000 7f5c1d15d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c18106160 0x7f5c18199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c1d15d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c1819a0a0 0x7f5c181a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c1d15d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5c181110d0 con 0x7f5c18106160 2024-04-23T18:36:48.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c1d15d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c18110f50 con 0x7f5c1819a0a0 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c1d15d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c18111250 con 0x7f5c18101090 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c16d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c18101090 0x7f5c181995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c1659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c18106160 0x7f5c18199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c16d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c18101090 0x7f5c181995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.132:33120/0 (socket says 172.21.15.132:33120) 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c1659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c18106160 0x7f5c18199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46828/0 (socket says 172.21.15.132:46828) 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c16d9d640 1 -- 172.21.15.132:0/2632198103 learned_addr learned my addr 172.21.15.132:0/2632198103 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c1759e640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c1819a0a0 0x7f5c181a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c16d9d640 1 -- 172.21.15.132:0/2632198103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c1819a0a0 msgr2=0x7f5c181a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c16d9d640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c1819a0a0 0x7f5c181a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c16d9d640 1 -- 172.21.15.132:0/2632198103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c18106160 msgr2=0x7f5c18199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c16d9d640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c18106160 0x7f5c18199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c16d9d640 1 -- 172.21.15.132:0/2632198103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c181a4e50 con 0x7f5c18101090 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c1659c640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c18106160 0x7f5c18199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:48.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.175+0000 7f5c16d9d640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c18101090 0x7f5c181995d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5c08005940 tx=0x7f5c08004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.176+0000 7f5bfffff640 1 -- 172.21.15.132:0/2632198103 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0803f030 con 0x7f5c18101090 2024-04-23T18:36:48.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.176+0000 7f5bfffff640 1 -- 172.21.15.132:0/2632198103 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c08003aa0 con 0x7f5c18101090 2024-04-23T18:36:48.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.176+0000 7f5bfffff640 1 -- 172.21.15.132:0/2632198103 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08035070 con 0x7f5c18101090 2024-04-23T18:36:48.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.176+0000 7f5c1d15d640 1 -- 172.21.15.132:0/2632198103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c181973c0 con 0x7f5c18101090 2024-04-23T18:36:48.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.176+0000 7f5c1d15d640 1 -- 172.21.15.132:0/2632198103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c18197830 con 0x7f5c18101090 2024-04-23T18:36:48.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.178+0000 7f5c1d15d640 1 -- 172.21.15.132:0/2632198103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c18102560 con 0x7f5c18101090 2024-04-23T18:36:48.171 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.180+0000 7f5bfffff640 1 -- 172.21.15.132:0/2632198103 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f5c08042030 con 0x7f5c18101090 2024-04-23T18:36:48.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.180+0000 7f5bfffff640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f5c000a3e40 0x7f5c000a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.180+0000 7f5bfffff640 1 -- 172.21.15.132:0/2632198103 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f5c080f1b70 con 0x7f5c18101090 2024-04-23T18:36:48.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.180+0000 7f5c1659c640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f5c000a3e40 0x7f5c000a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.181+0000 7f5c1659c640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f5c000a3e40 0x7f5c000a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c1806b0c0 tx=0x7f5c0c00b390 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.175 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.184+0000 7f5bfffff640 1 -- 172.21.15.132:0/2632198103 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c080be080 con 0x7f5c18101090 2024-04-23T18:36:48.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.483+0000 7f5c1d15d640 1 -- 172.21.15.132:0/2632198103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c180691c0 con 0x7f5c18101090 2024-04-23T18:36:48.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.484+0000 7f5bfffff640 1 -- 172.21.15.132:0/2632198103 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11946 (secure 0 0 0) 0x7f5c080bd980 con 0x7f5c18101090 2024-04-23T18:36:48.476 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:36:48.476 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:36:01.455959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24460,24425,24394,24491,24515]},"id":1}]} 2024-04-23T18:36:48.476 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 26 2024-04-23T18:36:48.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.487+0000 7f5c1d15d640 1 -- 172.21.15.132:0/2632198103 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f5c000a3e40 msgr2=0x7f5c000a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.487+0000 7f5c1d15d640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f5c000a3e40 0x7f5c000a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c1806b0c0 tx=0x7f5c0c00b390 comp rx=0 tx=0).stop 2024-04-23T18:36:48.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.487+0000 7f5c1d15d640 1 -- 172.21.15.132:0/2632198103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c18101090 msgr2=0x7f5c181995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.487+0000 7f5c1d15d640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c18101090 0x7f5c181995d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5c08005940 tx=0x7f5c08004270 comp rx=0 tx=0).stop 2024-04-23T18:36:48.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.487+0000 7f5c1d15d640 1 -- 172.21.15.132:0/2632198103 shutdown_connections 2024-04-23T18:36:48.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.487+0000 7f5c1d15d640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f5c000a3e40 0x7f5c000a6300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.487+0000 7f5c1d15d640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c1819a0a0 0x7f5c181a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.487+0000 7f5c1d15d640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c18106160 0x7f5c18199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.487+0000 7f5c1d15d640 1 --2- 172.21.15.132:0/2632198103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c18101090 0x7f5c181995d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.487+0000 7f5c1d15d640 1 -- 172.21.15.132:0/2632198103 >> 172.21.15.132:0/2632198103 conn(0x7f5c180771d0 msgr2=0x7f5c180fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.487+0000 7f5c1d15d640 1 -- 172.21.15.132:0/2632198103 shutdown_connections 2024-04-23T18:36:48.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.487+0000 7f5c1d15d640 1 -- 172.21.15.132:0/2632198103 wait complete. 2024-04-23T18:36:48.494 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24515 quiesce db / --timeout 60 --expiration 79.45 --await-for 42.0' 2024-04-23T18:36:48.495 DEBUG:teuthology.orchestra.run.smithi132:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24515 quiesce db / --timeout 60 --expiration 79.45 --await-for 42.0 2024-04-23T18:36:48.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.614+0000 7f97299ad640 1 -- 172.21.15.132:0/497056564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9724104d50 msgr2=0x7f9724108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.614+0000 7f97299ad640 1 --2- 172.21.15.132:0/497056564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9724104d50 0x7f9724108d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9718009860 tx=0x7f971802a340 comp rx=0 tx=0).stop 2024-04-23T18:36:48.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.614+0000 7f97299ad640 1 -- 172.21.15.132:0/497056564 shutdown_connections 2024-04-23T18:36:48.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.614+0000 7f97299ad640 1 --2- 172.21.15.132:0/497056564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9724104d50 0x7f9724108d90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.614+0000 7f97299ad640 1 --2- 172.21.15.132:0/497056564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9724109470 0x7f9724111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.614+0000 7f97299ad640 1 --2- 172.21.15.132:0/497056564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97241043a0 0x7f9724104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.614+0000 7f97299ad640 1 -- 172.21.15.132:0/497056564 >> 172.21.15.132:0/497056564 conn(0x7f97240ffc00 msgr2=0x7f9724102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.614+0000 7f97299ad640 1 -- 172.21.15.132:0/497056564 shutdown_connections 2024-04-23T18:36:48.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.614+0000 7f97299ad640 1 -- 172.21.15.132:0/497056564 wait complete. 2024-04-23T18:36:48.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.615+0000 7f97299ad640 1 Processor -- start 2024-04-23T18:36:48.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.615+0000 7f97299ad640 1 -- start start 2024-04-23T18:36:48.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.615+0000 7f97299ad640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97241043a0 0x7f97241a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.615+0000 7f97299ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9724109470 0x7f97241a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.615+0000 7f97299ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f972419a750 0x7f972419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.615+0000 7f97299ad640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f97241143b0 con 0x7f97241043a0 2024-04-23T18:36:48.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.615+0000 7f97299ad640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9724114230 con 0x7f9724109470 2024-04-23T18:36:48.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.615+0000 7f97299ad640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9724114530 con 0x7f972419a750 2024-04-23T18:36:48.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.615+0000 7f9722ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97241043a0 0x7f97241a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f9722ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97241043a0 0x7f97241a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46844/0 (socket says 172.21.15.132:46844) 2024-04-23T18:36:48.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f9722ffd640 1 -- 172.21.15.132:0/3557831129 learned_addr learned my addr 172.21.15.132:0/3557831129 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f97227fc640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9724109470 0x7f97241a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f97237fe640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f972419a750 0x7f972419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f9722ffd640 1 -- 172.21.15.132:0/3557831129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f972419a750 msgr2=0x7f972419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f9722ffd640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f972419a750 0x7f972419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f9722ffd640 1 -- 172.21.15.132:0/3557831129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9724109470 msgr2=0x7f97241a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f9722ffd640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9724109470 0x7f97241a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f9722ffd640 1 -- 172.21.15.132:0/3557831129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972419b350 con 0x7f97241043a0 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f97237fe640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f972419a750 0x7f972419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f9722ffd640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97241043a0 0x7f97241a0690 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f971400e9c0 tx=0x7f971400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f97289ab640 1 -- 172.21.15.132:0/3557831129 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971400cde0 con 0x7f97241043a0 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.616+0000 7f97299ad640 1 -- 172.21.15.132:0/3557831129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97241a7430 con 0x7f97241043a0 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.617+0000 7f97299ad640 1 -- 172.21.15.132:0/3557831129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97241a7830 con 0x7f97241043a0 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.617+0000 7f97289ab640 1 -- 172.21.15.132:0/3557831129 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97140035f0 con 0x7f97241043a0 2024-04-23T18:36:48.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.617+0000 7f97289ab640 1 -- 172.21.15.132:0/3557831129 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714010690 con 0x7f97241043a0 2024-04-23T18:36:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.619+0000 7f97289ab640 1 -- 172.21.15.132:0/3557831129 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f9714003aa0 con 0x7f97241043a0 2024-04-23T18:36:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.620+0000 7f97289ab640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f970c0a3e40 0x7f970c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.620+0000 7f97289ab640 1 -- 172.21.15.132:0/3557831129 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f97140d0de0 con 0x7f97241043a0 2024-04-23T18:36:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.620+0000 7f97227fc640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f970c0a3e40 0x7f970c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.620+0000 7f97227fc640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f970c0a3e40 0x7f970c0a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9718009830 tx=0x7f97180058d0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.620+0000 7f972c7a5740 1 Processor -- start 2024-04-23T18:36:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.620+0000 7f972c7a5740 1 -- start start 2024-04-23T18:36:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f972c7a5740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0a4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f972c7a5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0a49c0 0x556c7c0aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f972c7a5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0af090 0x556c7c0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f972c7a5740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x556c7c009370 con 0x556c7c0a40a0 2024-04-23T18:36:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f972c7a5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556c7bfa62e0 con 0x556c7c0a49c0 2024-04-23T18:36:48.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f972c7a5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556c7c005330 con 0x556c7c0af090 2024-04-23T18:36:48.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f9722ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f9722ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0a4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46846/0 (socket says 172.21.15.132:46846) 2024-04-23T18:36:48.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f9722ffd640 1 -- 172.21.15.132:0/1740046958 learned_addr learned my addr 172.21.15.132:0/1740046958 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:48.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f97227fc640 1 --2- 172.21.15.132:0/1740046958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0a49c0 0x556c7c0aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f97237fe640 1 --2- 172.21.15.132:0/1740046958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0af090 0x556c7c0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f9722ffd640 1 -- 172.21.15.132:0/1740046958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0af090 msgr2=0x556c7c0b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f9722ffd640 1 --2- 172.21.15.132:0/1740046958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0af090 0x556c7c0b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f9722ffd640 1 -- 172.21.15.132:0/1740046958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0a49c0 msgr2=0x556c7c0aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f9722ffd640 1 --2- 172.21.15.132:0/1740046958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0a49c0 0x556c7c0aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f9722ffd640 1 -- 172.21.15.132:0/1740046958 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c7bde7d80 con 0x556c7c0a40a0 2024-04-23T18:36:48.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f97237fe640 1 --2- 172.21.15.132:0/1740046958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0af090 0x556c7c0b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:48.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f9722ffd640 1 --2- 172.21.15.132:0/1740046958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0a4480 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9714000c00 tx=0x7f971409d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c328a650374fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.621+0000 7f9709ffb640 1 -- 172.21.15.132:0/1740046958 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97140b6070 con 0x556c7c0a40a0 2024-04-23T18:36:48.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.622+0000 7f9709ffb640 1 -- 172.21.15.132:0/1740046958 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971409de50 con 0x556c7c0a40a0 2024-04-23T18:36:48.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.622+0000 7f9709ffb640 1 -- 172.21.15.132:0/1740046958 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97140038c0 con 0x556c7c0a40a0 2024-04-23T18:36:48.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.622+0000 7f972c7a5740 1 -- 172.21.15.132:0/1740046958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 msgr2=0x556c7c0a4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.622+0000 7f972c7a5740 1 --2- 172.21.15.132:0/1740046958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0a4480 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9714000c00 tx=0x7f971409d2e0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.622+0000 7f972c7a5740 1 -- 172.21.15.132:0/1740046958 shutdown_connections 2024-04-23T18:36:48.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.623+0000 7f972c7a5740 1 --2- 172.21.15.132:0/1740046958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0af090 0x556c7c0b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.623+0000 7f972c7a5740 1 --2- 172.21.15.132:0/1740046958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0a49c0 0x556c7c0aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.623+0000 7f972c7a5740 1 --2- 172.21.15.132:0/1740046958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0a4480 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.623+0000 7f972c7a5740 1 -- 172.21.15.132:0/1740046958 >> 172.21.15.132:0/1740046958 conn(0x556c7c0a03c0 msgr2=0x556c7c0a07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.623+0000 7f972c7a5740 1 -- 172.21.15.132:0/1740046958 shutdown_connections 2024-04-23T18:36:48.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.623+0000 7f972c7a5740 1 -- 172.21.15.132:0/1740046958 wait complete. 2024-04-23T18:36:48.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.623+0000 7f972c7a5740 1 -- start start 2024-04-23T18:36:48.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.623+0000 7f972c7a5740 1 Processor -- start 2024-04-23T18:36:48.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f972c7a5740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f972c7a5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0a49c0 0x556c7c0ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f972c7a5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0af090 0x556c7c0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f9722ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f972c7a5740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x556c7c0b1e80 con 0x556c7c0a40a0 2024-04-23T18:36:48.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f9722ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46854/0 (socket says 172.21.15.132:46854) 2024-04-23T18:36:48.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f9722ffd640 1 -- 172.21.15.132:0/414808407 learned_addr learned my addr 172.21.15.132:0/414808407 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:48.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f972c7a5740 1 -- 172.21.15.132:0/414808407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556c7bfa3fb0 con 0x556c7c0af090 2024-04-23T18:36:48.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f972c7a5740 1 -- 172.21.15.132:0/414808407 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556c7c0b2000 con 0x556c7c0a49c0 2024-04-23T18:36:48.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f97237fe640 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0af090 0x556c7c0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f97227fc640 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0a49c0 0x556c7c0ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f972c7a5740 10 client.0 _ll_register_callbacks cb 0x556c7bfa39a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:48.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f9722ffd640 1 -- 172.21.15.132:0/414808407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0a49c0 msgr2=0x556c7c0ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f9722ffd640 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0a49c0 0x556c7c0ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f9722ffd640 1 -- 172.21.15.132:0/414808407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0af090 msgr2=0x556c7c0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f9722ffd640 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0af090 0x556c7c0acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f9722ffd640 1 -- 172.21.15.132:0/414808407 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c7be48510 con 0x556c7c0a40a0 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f972c7a5740 1 -- 172.21.15.132:0/414808407 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f971409de50 con 0x556c7c0a40a0 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f9722ffd640 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0ac090 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f97140994f0 tx=0x7f971409ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea40a8aacd03c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.624+0000 7f9702ffd640 10 client.15792 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.625+0000 7f9702ffd640 1 -- 172.21.15.132:0/414808407 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97140b6040 con 0x556c7c0a40a0 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.625+0000 7f9702ffd640 1 -- 172.21.15.132:0/414808407 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f9714003880 con 0x556c7c0a40a0 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.625+0000 7f9702ffd640 1 -- 172.21.15.132:0/414808407 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f97140a5790 con 0x556c7c0a40a0 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.625+0000 7f972c7a5740 10 client.15792 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.625+0000 7f972c7a5740 1 -- 172.21.15.132:0/414808407 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({fsmap=26}) -- 0x556c7bcdb670 con 0x556c7c0a40a0 2024-04-23T18:36:48.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.625+0000 7f9702ffd640 1 -- 172.21.15.132:0/414808407 <== mon.0 v2:172.21.15.132:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f97140bbce0 con 0x556c7c0a40a0 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.625+0000 7f972c7a5740 10 client.15792 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.625+0000 7f972c7a5740 10 client.15792 resolve_mds: validated gid 24515 aka daemon mds.b 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.625+0000 7f972c7a5740 20 client.15792 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.625+0000 7f972c7a5740 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x556c7c0bf170 0x556c7c0c1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.625+0000 7f972c7a5740 4 client.15792 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.625+0000 7f972c7a5740 1 -- 172.21.15.132:0/414808407 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c7c016d90 con 0x556c7c0bf170 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.626+0000 7f97227fc640 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x556c7c0bf170 0x556c7c0c1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.626+0000 7f97227fc640 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x556c7c0bf170 0x556c7c0c1560 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e32cb0290913a203 server_cookie=996a5217c0c06bd9 in_seq=0 out_seq=0 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.626+0000 7f9702ffd640 10 client.15792 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f9702ffd640 1 -- 172.21.15.132:0/414808407 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c7c016d90 con 0x556c7c0bf170 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f9702ffd640 10 client.15792 handle_command_reply: tid=0 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f972c7a5740 1 client.15792 shutdown 2024-04-23T18:36:48.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f972c7a5740 1 -- 172.21.15.132:0/414808407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 msgr2=0x556c7c0ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f972c7a5740 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0ac090 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f97140994f0 tx=0x7f971409ca00 comp rx=0 tx=0).stop 2024-04-23T18:36:48.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f972c7a5740 1 -- 172.21.15.132:0/414808407 shutdown_connections 2024-04-23T18:36:48.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f972c7a5740 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x556c7c0bf170 0x556c7c0c1560 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f9702ffd640 0 client.15792 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:48.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f972c7a5740 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0a49c0 0x556c7c0ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f972c7a5740 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0af090 0x556c7c0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f972c7a5740 1 --2- 172.21.15.132:0/414808407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0ac090 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f972c7a5740 1 -- 172.21.15.132:0/414808407 >> 172.21.15.132:0/414808407 conn(0x556c7c0a03c0 msgr2=0x556c7c09f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f972c7a5740 1 -- 172.21.15.132:0/414808407 shutdown_connections 2024-04-23T18:36:48.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f972c7a5740 1 -- 172.21.15.132:0/414808407 wait complete. 2024-04-23T18:36:48.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.627+0000 7f972c7a5740 20 client.15792 trim_cache size 0 max 16384 2024-04-23T18:36:48.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f972c7a5740 1 Processor -- start 2024-04-23T18:36:48.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f972c7a5740 1 -- start start 2024-04-23T18:36:48.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f972c7a5740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0c4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f972c7a5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0a49c0 0x556c7c0c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f972c7a5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0af090 0x556c7c0a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f972c7a5740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f97241aba50 con 0x556c7c0a40a0 2024-04-23T18:36:48.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f972c7a5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97241ad590 con 0x556c7c0af090 2024-04-23T18:36:48.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f972c7a5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f97241ac120 con 0x556c7c0a49c0 2024-04-23T18:36:48.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f9722ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f9722ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0c4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46862/0 (socket says 172.21.15.132:46862) 2024-04-23T18:36:48.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f9722ffd640 1 -- 172.21.15.132:0/3096604779 learned_addr learned my addr 172.21.15.132:0/3096604779 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:48.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f97227fc640 1 --2- 172.21.15.132:0/3096604779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0a49c0 0x556c7c0c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f97237fe640 1 --2- 172.21.15.132:0/3096604779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0af090 0x556c7c0a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f9722ffd640 1 -- 172.21.15.132:0/3096604779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0a49c0 msgr2=0x556c7c0c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f9722ffd640 1 --2- 172.21.15.132:0/3096604779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0a49c0 0x556c7c0c4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f9722ffd640 1 -- 172.21.15.132:0/3096604779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0af090 msgr2=0x556c7c0a92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f9722ffd640 1 --2- 172.21.15.132:0/3096604779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0af090 0x556c7c0a92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.660+0000 7f9722ffd640 1 -- 172.21.15.132:0/3096604779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c7c0a5560 con 0x556c7c0a40a0 2024-04-23T18:36:48.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.661+0000 7f9722ffd640 1 --2- 172.21.15.132:0/3096604779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0c4590 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f97140ceb20 tx=0x7f97140d0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7568c29b8bdf9420 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.661+0000 7f97097fa640 1 -- 172.21.15.132:0/3096604779 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971409c8f0 con 0x556c7c0a40a0 2024-04-23T18:36:48.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.661+0000 7f97097fa640 1 -- 172.21.15.132:0/3096604779 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971409de50 con 0x556c7c0a40a0 2024-04-23T18:36:48.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.661+0000 7f97097fa640 1 -- 172.21.15.132:0/3096604779 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971409bbe0 con 0x556c7c0a40a0 2024-04-23T18:36:48.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.662+0000 7f972c7a5740 1 -- 172.21.15.132:0/3096604779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 msgr2=0x556c7c0c4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.662+0000 7f972c7a5740 1 --2- 172.21.15.132:0/3096604779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0c4590 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f97140ceb20 tx=0x7f97140d0000 comp rx=0 tx=0).stop 2024-04-23T18:36:48.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.662+0000 7f972c7a5740 1 -- 172.21.15.132:0/3096604779 shutdown_connections 2024-04-23T18:36:48.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.662+0000 7f972c7a5740 1 --2- 172.21.15.132:0/3096604779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0a49c0 0x556c7c0c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.662+0000 7f972c7a5740 1 --2- 172.21.15.132:0/3096604779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0af090 0x556c7c0a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.662+0000 7f972c7a5740 1 --2- 172.21.15.132:0/3096604779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a40a0 0x556c7c0c4590 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.662+0000 7f972c7a5740 1 -- 172.21.15.132:0/3096604779 >> 172.21.15.132:0/3096604779 conn(0x556c7c0a03c0 msgr2=0x556c7c0a6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.662+0000 7f972c7a5740 1 -- 172.21.15.132:0/3096604779 shutdown_connections 2024-04-23T18:36:48.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.662+0000 7f972c7a5740 1 -- 172.21.15.132:0/3096604779 wait complete. 2024-04-23T18:36:48.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.662+0000 7f972c7a5740 1 -- start start 2024-04-23T18:36:48.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f972c7a5740 1 Processor -- start 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f972c7a5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0a40a0 0x556c7c1194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f972c7a5740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a49c0 0x556c7c119a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f972c7a5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0af090 0x556c7c124360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f972c7a5740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f97241acec0 con 0x556c7c0a49c0 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f972c7a5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97241ac7f0 con 0x556c7c0a40a0 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f972c7a5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f97241a9a30 con 0x556c7c0af090 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f972c7a5740 10 client.0 _ll_register_callbacks cb 0x7f9714098e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f97227fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a49c0 0x556c7c119a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f97227fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a49c0 0x556c7c119a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46876/0 (socket says 172.21.15.132:46876) 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f97227fc640 1 -- 172.21.15.132:0/576701113 learned_addr learned my addr 172.21.15.132:0/576701113 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f97237fe640 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0af090 0x556c7c124360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f9722ffd640 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0a40a0 0x556c7c1194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f97227fc640 1 -- 172.21.15.132:0/576701113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0af090 msgr2=0x556c7c124360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f97227fc640 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0af090 0x556c7c124360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f97227fc640 1 -- 172.21.15.132:0/576701113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0a40a0 msgr2=0x556c7c1194d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f97227fc640 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0a40a0 0x556c7c1194d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f97227fc640 1 -- 172.21.15.132:0/576701113 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c7c124950 con 0x556c7c0a49c0 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f9722ffd640 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0a40a0 0x556c7c1194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.663+0000 7f97237fe640 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0af090 0x556c7c124360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f97227fc640 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a49c0 0x556c7c119a10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9718000c00 tx=0x7f971802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c62e3c4cf51688 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f9702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f9702ffd640 1 -- 172.21.15.132:0/576701113 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971805b020 con 0x556c7c0a49c0 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f9702ffd640 1 -- 172.21.15.132:0/576701113 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f97180307d0 con 0x556c7c0a49c0 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f972c7a5740 1 -- 172.21.15.132:0/576701113 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f971409de50 con 0x556c7c0a49c0 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f9702ffd640 1 -- 172.21.15.132:0/576701113 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9718066050 con 0x556c7c0a49c0 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f972c7a5740 10 client.15798 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f972c7a5740 1 -- 172.21.15.132:0/576701113 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556c7c125c70 con 0x556c7c0a49c0 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f9702ffd640 1 -- 172.21.15.132:0/576701113 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971409de50 con 0x556c7c0a49c0 2024-04-23T18:36:48.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f9702ffd640 1 -- 172.21.15.132:0/576701113 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9718003c50 con 0x556c7c0a49c0 2024-04-23T18:36:48.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f972c7a5740 10 client.15798 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:48.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f972c7a5740 10 client.15798 resolve_mds: validated gid 24515 aka daemon mds.b 2024-04-23T18:36:48.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f972c7a5740 20 client.15798 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:48.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f972c7a5740 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x556c7c0bf170 0x556c7c128440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:48.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f972c7a5740 4 client.15798 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.45, "await_for": 42.0}] 2024-04-23T18:36:48.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.664+0000 7f972c7a5740 1 -- 172.21.15.132:0/576701113 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.45, "await_for": 42.0}) -- 0x556c7c010c00 con 0x556c7c0bf170 2024-04-23T18:36:48.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.665+0000 7f9722ffd640 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x556c7c0bf170 0x556c7c128440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:48.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.665+0000 7f9702ffd640 10 client.15798 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:48.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.665+0000 7f9722ffd640 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x556c7c0bf170 0x556c7c128440 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0382c320edd0dfd server_cookie=efde760a2cb90512 in_seq=0 out_seq=0 2024-04-23T18:36:48.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.673+0000 7f9702ffd640 1 -- 172.21.15.132:0/576701113 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x556c7c010c00 con 0x556c7c0bf170 2024-04-23T18:36:48.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.673+0000 7f9702ffd640 10 client.15798 handle_command_reply: tid=0 2024-04-23T18:36:48.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.673+0000 7f972c7a5740 1 client.15798 shutdown 2024-04-23T18:36:48.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f972c7a5740 1 -- 172.21.15.132:0/576701113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a49c0 msgr2=0x556c7c119a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f972c7a5740 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a49c0 0x556c7c119a10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9718000c00 tx=0x7f971802a340 comp rx=0 tx=0).stop 2024-04-23T18:36:48.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f972c7a5740 1 -- 172.21.15.132:0/576701113 shutdown_connections 2024-04-23T18:36:48.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f972c7a5740 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x556c7c0bf170 0x556c7c128440 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f972c7a5740 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c7c0af090 0x556c7c124360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f972c7a5740 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c7c0a49c0 0x556c7c119a10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f9702ffd640 0 client.15798 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:48.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f972c7a5740 1 --2- 172.21.15.132:0/576701113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c7c0a40a0 0x556c7c1194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f972c7a5740 1 -- 172.21.15.132:0/576701113 >> 172.21.15.132:0/576701113 conn(0x556c7c0a03c0 msgr2=0x556c7c09ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f972c7a5740 1 -- 172.21.15.132:0/576701113 shutdown_connections 2024-04-23T18:36:48.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f972c7a5740 1 -- 172.21.15.132:0/576701113 wait complete. 2024-04-23T18:36:48.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f972c7a5740 20 client.15798 trim_cache size 0 max 16384 2024-04-23T18:36:48.667 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:36:48.667 INFO:teuthology.orchestra.run.smithi132.stdout: "epoch": 26, 2024-04-23T18:36:48.667 INFO:teuthology.orchestra.run.smithi132.stdout: "leader": 24515, 2024-04-23T18:36:48.667 INFO:teuthology.orchestra.run.smithi132.stdout: "set_version": 6, 2024-04-23T18:36:48.667 INFO:teuthology.orchestra.run.smithi132.stdout: "sets": { 2024-04-23T18:36:48.667 INFO:teuthology.orchestra.run.smithi132.stdout: "320f00aa": { 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "version": 6, 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "age_ref": 0.0, 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "state": { 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "QUIESCED", 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "age": 0.0 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "timeout": 60.0, 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "expiration": 79.5, 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "members": { 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "file:/": { 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "excluded": false, 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "state": { 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "QUIESCED", 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: "age": 0.0 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:48.668 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:48.669 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:48.669 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:48.669 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:36:48.669 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:36:48.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f97299ad640 1 -- 172.21.15.132:0/3557831129 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f970c0a3e40 msgr2=0x7f970c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f97299ad640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f970c0a3e40 0x7f970c0a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9718009830 tx=0x7f97180058d0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f97299ad640 1 -- 172.21.15.132:0/3557831129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97241043a0 msgr2=0x7f97241a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:48.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.674+0000 7f97299ad640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97241043a0 0x7f97241a0690 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f971400e9c0 tx=0x7f971400ee80 comp rx=0 tx=0).stop 2024-04-23T18:36:48.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.675+0000 7f97299ad640 1 -- 172.21.15.132:0/3557831129 shutdown_connections 2024-04-23T18:36:48.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.675+0000 7f97299ad640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f970c0a3e40 0x7f970c0a6300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.675+0000 7f97299ad640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f972419a750 0x7f972419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.675+0000 7f97299ad640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9724109470 0x7f97241a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.675+0000 7f97299ad640 1 --2- 172.21.15.132:0/3557831129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97241043a0 0x7f97241a0690 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:48.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.675+0000 7f97299ad640 1 -- 172.21.15.132:0/3557831129 >> 172.21.15.132:0/3557831129 conn(0x7f97240ffc00 msgr2=0x7f9724100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:48.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.675+0000 7f97299ad640 1 -- 172.21.15.132:0/3557831129 shutdown_connections 2024-04-23T18:36:48.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:36:48.675+0000 7f97299ad640 1 -- 172.21.15.132:0/3557831129 wait complete. 2024-04-23T18:36:48.678 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '320f00aa', quiesce took 0.6 seconds. Will release after: 19.45 2024-04-23T18:36:48.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:48 smithi132 ceph-mon[29178]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T18:36:48.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:48 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/686133450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:48.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:48 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2632198103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:48 smithi148 ceph-mon[32311]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T18:36:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:48 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/686133450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:48 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2632198103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:48 smithi157 ceph-mon[32580]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T18:36:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:48 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/686133450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:49.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:48 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2632198103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:50.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:50 smithi132 ceph-mon[29178]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-23T18:36:51.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:50 smithi148 ceph-mon[32311]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-23T18:36:51.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:50 smithi157 ceph-mon[32580]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-23T18:36:52.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:52 smithi132 ceph-mon[29178]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:36:53.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:52 smithi148 ceph-mon[32311]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:36:53.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:52 smithi157 ceph-mon[32580]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:36:54.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:54 smithi132 ceph-mon[29178]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:36:55.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:54 smithi148 ceph-mon[32311]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:36:55.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:54 smithi157 ceph-mon[32580]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:36:56.250 INFO:tasks.workunit.client.0.smithi132.stderr:Note: switching to '57e0caf76fdb7ab8b1358588f292d08519163844'. 2024-04-23T18:36:56.250 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-04-23T18:36:56.250 INFO:tasks.workunit.client.0.smithi132.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-23T18:36:56.250 INFO:tasks.workunit.client.0.smithi132.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr:state without impacting any branches by switching back to a branch. 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr: git switch -c 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr:Or undo this operation with: 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr: git switch - 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-04-23T18:36:56.251 INFO:tasks.workunit.client.0.smithi132.stderr:HEAD is now at 57e0caf76fd qa/suites/fs: add quiescer to the fs suite 2024-04-23T18:36:56.252 DEBUG:teuthology.orchestra.run.smithi132:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-23T18:36:56.275 INFO:tasks.workunit.client.0.smithi132.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-23T18:36:56.277 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T18:36:56.277 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-23T18:36:56.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:56 smithi132 ceph-mon[29178]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:36:56.890 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-23T18:36:56.960 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-23T18:36:57.001 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T18:36:57.002 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T18:36:57.002 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-23T18:36:57.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:56 smithi148 ceph-mon[32311]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:36:57.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:56 smithi157 ceph-mon[32580]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:36:57.043 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T18:36:57.047 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:36:57.047 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-23T18:36:57.102 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-04-23T18:36:57.104 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-04-23T18:36:57.104 DEBUG:teuthology.orchestra.run.smithi132:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57e0caf76fdb7ab8b1358588f292d08519163844 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-04-23T18:36:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.041+0000 7fb2594d0640 1 -- 172.21.15.132:0/1872766930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb254107330 msgr2=0x7fb25410eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.041+0000 7fb2594d0640 1 --2- 172.21.15.132:0/1872766930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb254107330 0x7fb25410eeb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009900 tx=0x7fb24c02a410 comp rx=0 tx=0).stop 2024-04-23T18:36:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.041+0000 7fb2594d0640 1 -- 172.21.15.132:0/1872766930 shutdown_connections 2024-04-23T18:36:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.041+0000 7fb2594d0640 1 --2- 172.21.15.132:0/1872766930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb254107330 0x7fb25410eeb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.041+0000 7fb2594d0640 1 --2- 172.21.15.132:0/1872766930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb254101070 0x7fb254106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.041+0000 7fb2594d0640 1 --2- 172.21.15.132:0/1872766930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2541006c0 0x7fb254100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.041+0000 7fb2594d0640 1 -- 172.21.15.132:0/1872766930 >> 172.21.15.132:0/1872766930 conn(0x7fb254077390 msgr2=0x7fb2540fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.041+0000 7fb2594d0640 1 -- 172.21.15.132:0/1872766930 shutdown_connections 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.041+0000 7fb2594d0640 1 -- 172.21.15.132:0/1872766930 wait complete. 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb2594d0640 1 Processor -- start 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb2594d0640 1 -- start start 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb2594d0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2541006c0 0x7fb254198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb2594d0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb254107330 0x7fb254198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb2594d0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25419cf80 0x7fb25419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb2594d0640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb254112250 con 0x7fb2541006c0 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb2594d0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541120d0 con 0x7fb254107330 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb2594d0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541123d0 con 0x7fb25419cf80 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb252ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2541006c0 0x7fb254198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb252ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2541006c0 0x7fb254198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56112/0 (socket says 172.21.15.132:56112) 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb252ffd640 1 -- 172.21.15.132:0/612670475 learned_addr learned my addr 172.21.15.132:0/612670475 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb2527fc640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb254107330 0x7fb254198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb2537fe640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25419cf80 0x7fb25419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb252ffd640 1 -- 172.21.15.132:0/612670475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25419cf80 msgr2=0x7fb25419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb252ffd640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25419cf80 0x7fb25419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb252ffd640 1 -- 172.21.15.132:0/612670475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb254107330 msgr2=0x7fb254198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb252ffd640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb254107330 0x7fb254198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb252ffd640 1 -- 172.21.15.132:0/612670475 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25419da70 con 0x7fb2541006c0 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb2527fc640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb254107330 0x7fb254198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.042+0000 7fb2537fe640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25419cf80 0x7fb25419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.043+0000 7fb252ffd640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2541006c0 0x7fb254198710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb24400cce0 tx=0x7fb24400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.043+0000 7fb24bfff640 1 -- 172.21.15.132:0/612670475 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244004b20 con 0x7fb2541006c0 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.043+0000 7fb24bfff640 1 -- 172.21.15.132:0/612670475 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb244004cc0 con 0x7fb2541006c0 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.043+0000 7fb2594d0640 1 -- 172.21.15.132:0/612670475 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25419dd60 con 0x7fb2541006c0 2024-04-23T18:36:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.043+0000 7fb24bfff640 1 -- 172.21.15.132:0/612670475 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24400f7c0 con 0x7fb2541006c0 2024-04-23T18:36:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.043+0000 7fb2594d0640 1 -- 172.21.15.132:0/612670475 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2541a5500 con 0x7fb2541006c0 2024-04-23T18:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.045+0000 7fb24bfff640 1 -- 172.21.15.132:0/612670475 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fb244005560 con 0x7fb2541006c0 2024-04-23T18:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.045+0000 7fb25c1e2740 1 Processor -- start 2024-04-23T18:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.045+0000 7fb25c1e2740 1 -- start start 2024-04-23T18:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.045+0000 7fb24bfff640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fb2400a3ee0 0x7fb2400a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.045+0000 7fb25c1e2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4439fd620 0x55b4439fda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb25c1e2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb25c1e2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b443a08800 0x55b443a0abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb25c1e2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b443962970 con 0x55b4439fdf40 2024-04-23T18:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb25c1e2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b4438ff4a0 con 0x55b443a08800 2024-04-23T18:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb25c1e2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b44395e930 con 0x55b4439fd620 2024-04-23T18:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2527fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2537fe640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fb2400a3ee0 0x7fb2400a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2527fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56128/0 (socket says 172.21.15.132:56128) 2024-04-23T18:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb24bfff640 1 -- 172.21.15.132:0/612670475 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fb2440d0c30 con 0x7fb2541006c0 2024-04-23T18:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2527fc640 1 -- 172.21.15.132:0/212622982 learned_addr learned my addr 172.21.15.132:0/212622982 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2537fe640 1 --2- 172.21.15.132:0/212622982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b443a08800 0x55b443a0abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb252ffd640 1 --2- 172.21.15.132:0/212622982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4439fd620 0x55b4439fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2527fc640 1 -- 172.21.15.132:0/212622982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4439fd620 msgr2=0x55b4439fda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2527fc640 1 --2- 172.21.15.132:0/212622982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4439fd620 0x55b4439fda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2527fc640 1 -- 172.21.15.132:0/212622982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b443a08800 msgr2=0x55b443a0abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2527fc640 1 --2- 172.21.15.132:0/212622982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b443a08800 0x55b443a0abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2527fc640 1 -- 172.21.15.132:0/212622982 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b443675210 con 0x55b4439fdf40 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2537fe640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fb2400a3ee0 0x7fb2400a63a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb24c006010 tx=0x7fb24c0314a0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2527fc640 1 --2- 172.21.15.132:0/212622982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a082c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009b30 tx=0x7fb23c00e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5182e2f389f98f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2497fa640 1 -- 172.21.15.132:0/212622982 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c016070 con 0x55b4439fdf40 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2497fa640 1 -- 172.21.15.132:0/212622982 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb23c00ed20 con 0x55b4439fdf40 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.046+0000 7fb2497fa640 1 -- 172.21.15.132:0/212622982 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c010630 con 0x55b4439fdf40 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.047+0000 7fb25c1e2740 1 -- 172.21.15.132:0/212622982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 msgr2=0x55b443a082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.047+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/212622982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a082c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009b30 tx=0x7fb23c00e2f0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.047+0000 7fb25c1e2740 1 -- 172.21.15.132:0/212622982 shutdown_connections 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.047+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/212622982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b443a08800 0x55b443a0abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.047+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/212622982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a082c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.047+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/212622982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4439fd620 0x55b4439fda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.047+0000 7fb25c1e2740 1 -- 172.21.15.132:0/212622982 >> 172.21.15.132:0/212622982 conn(0x55b4439f9b30 msgr2=0x55b4439f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.047+0000 7fb25c1e2740 1 -- 172.21.15.132:0/212622982 shutdown_connections 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.047+0000 7fb25c1e2740 1 -- 172.21.15.132:0/212622982 wait complete. 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.047+0000 7fb25c1e2740 1 -- start start 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb25c1e2740 1 Processor -- start 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb25c1e2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b443a08170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb25c1e2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a05460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb25c1e2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 0x55b443a059c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb25c1e2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b443a0b430 con 0x55b4439fdf40 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb25c1e2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b4438fd0b0 con 0x55b4439fd620 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb25c1e2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b443a0b5b0 con 0x55b443a08800 2024-04-23T18:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb25c1e2740 10 client.0 _ll_register_callbacks cb 0x55b4438ead30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb2527fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a05460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb2527fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a05460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56134/0 (socket says 172.21.15.132:56134) 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb2527fc640 1 -- 172.21.15.132:0/862383503 learned_addr learned my addr 172.21.15.132:0/862383503 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb252ffd640 1 --2- 172.21.15.132:0/862383503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b443a08170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb2527fc640 1 -- 172.21.15.132:0/862383503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 msgr2=0x55b443a059c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb2527fc640 1 --2- 172.21.15.132:0/862383503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 0x55b443a059c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb2527fc640 1 -- 172.21.15.132:0/862383503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 msgr2=0x55b443a08170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb2527fc640 1 --2- 172.21.15.132:0/862383503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b443a08170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb2527fc640 1 -- 172.21.15.132:0/862383503 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b443741620 con 0x55b4439fdf40 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb2527fc640 1 --2- 172.21.15.132:0/862383503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a05460 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb23c005c90 tx=0x7fb23c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8617cac36fa4fc1a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb23a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb23a7fc640 1 -- 172.21.15.132:0/862383503 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c028060 con 0x55b4439fdf40 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb23a7fc640 1 -- 172.21.15.132:0/862383503 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fb23c0048d0 con 0x55b4439fdf40 2024-04-23T18:36:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.048+0000 7fb25c1e2740 1 -- 172.21.15.132:0/862383503 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb23c00ed20 con 0x55b4439fdf40 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.049+0000 7fb23a7fc640 1 -- 172.21.15.132:0/862383503 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb23c016070 con 0x55b4439fdf40 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.049+0000 7fb25c1e2740 10 client.15810 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.049+0000 7fb25c1e2740 1 -- 172.21.15.132:0/862383503 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b443a172d0 con 0x55b4439fdf40 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.049+0000 7fb23a7fc640 1 -- 172.21.15.132:0/862383503 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb23c00ed20 con 0x55b4439fdf40 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.049+0000 7fb23a7fc640 1 -- 172.21.15.132:0/862383503 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb23c00f040 con 0x55b4439fdf40 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.049+0000 7fb25c1e2740 10 client.15810 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.049+0000 7fb25c1e2740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.049+0000 7fb25c1e2740 20 client.15810 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.049+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/862383503 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b443a18730 0x55b443a1ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.049+0000 7fb25c1e2740 4 client.15810 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.049+0000 7fb25c1e2740 1 -- 172.21.15.132:0/862383503 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b443970390 con 0x55b443a18730 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.049+0000 7fb252ffd640 1 --2- 172.21.15.132:0/862383503 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b443a18730 0x55b443a1ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.050+0000 7fb23a7fc640 10 client.15810 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.050+0000 7fb252ffd640 1 --2- 172.21.15.132:0/862383503 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b443a18730 0x55b443a1ab20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1dfaceb8027d5ee server_cookie=cc1fe21e51f05a30 in_seq=0 out_seq=0 2024-04-23T18:36:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.050+0000 7fb23a7fc640 1 -- 172.21.15.132:0/862383503 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b443970390 con 0x55b443a18730 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.050+0000 7fb23a7fc640 10 client.15810 handle_command_reply: tid=0 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.050+0000 7fb25c1e2740 1 client.15810 shutdown 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.051+0000 7fb25c1e2740 1 -- 172.21.15.132:0/862383503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 msgr2=0x55b443a05460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.051+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/862383503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a05460 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb23c005c90 tx=0x7fb23c005270 comp rx=0 tx=0).stop 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.051+0000 7fb25c1e2740 1 -- 172.21.15.132:0/862383503 shutdown_connections 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.051+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/862383503 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b443a18730 0x55b443a1ab20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.051+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/862383503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 0x55b443a059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.051+0000 7fb23a7fc640 0 client.15810 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.051+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/862383503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a05460 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.051+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/862383503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b443a08170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.051+0000 7fb25c1e2740 1 -- 172.21.15.132:0/862383503 >> 172.21.15.132:0/862383503 conn(0x55b4439f9b30 msgr2=0x55b4439ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.051+0000 7fb25c1e2740 1 -- 172.21.15.132:0/862383503 shutdown_connections 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.051+0000 7fb25c1e2740 1 -- 172.21.15.132:0/862383503 wait complete. 2024-04-23T18:36:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.051+0000 7fb25c1e2740 20 client.15810 trim_cache size 0 max 16384 2024-04-23T18:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.065+0000 7fb25c1e2740 1 Processor -- start 2024-04-23T18:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.065+0000 7fb25c1e2740 1 -- start start 2024-04-23T18:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.065+0000 7fb25c1e2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b4439f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.065+0000 7fb25c1e2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a07ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.065+0000 7fb25c1e2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 0x55b443a10800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.065+0000 7fb25c1e2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541adbd0 con 0x55b4439fdf40 2024-04-23T18:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.065+0000 7fb25c1e2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541af710 con 0x55b4439fd620 2024-04-23T18:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.065+0000 7fb25c1e2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541ae2a0 con 0x55b443a08800 2024-04-23T18:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.065+0000 7fb2527fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a07ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.065+0000 7fb2527fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a07ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56138/0 (socket says 172.21.15.132:56138) 2024-04-23T18:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.065+0000 7fb2527fc640 1 -- 172.21.15.132:0/2279600909 learned_addr learned my addr 172.21.15.132:0/2279600909 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.066+0000 7fb2537fe640 1 --2- 172.21.15.132:0/2279600909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 0x55b443a10800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.066+0000 7fb252ffd640 1 --2- 172.21.15.132:0/2279600909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b4439f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.066+0000 7fb2527fc640 1 -- 172.21.15.132:0/2279600909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 msgr2=0x55b443a10800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.066+0000 7fb2527fc640 1 --2- 172.21.15.132:0/2279600909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 0x55b443a10800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.066+0000 7fb2527fc640 1 -- 172.21.15.132:0/2279600909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 msgr2=0x55b4439f8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.066+0000 7fb2527fc640 1 --2- 172.21.15.132:0/2279600909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b4439f8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.066+0000 7fb2527fc640 1 -- 172.21.15.132:0/2279600909 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4439fdd70 con 0x55b4439fdf40 2024-04-23T18:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.066+0000 7fb252ffd640 1 --2- 172.21.15.132:0/2279600909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b4439f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.066+0000 7fb2527fc640 1 --2- 172.21.15.132:0/2279600909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a07ca0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb23c005d80 tx=0x7fb23c009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674a44205c55c371 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.066+0000 7fb248ff9640 1 -- 172.21.15.132:0/2279600909 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c028030 con 0x55b4439fdf40 2024-04-23T18:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.066+0000 7fb248ff9640 1 -- 172.21.15.132:0/2279600909 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb23c01bad0 con 0x55b4439fdf40 2024-04-23T18:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.066+0000 7fb248ff9640 1 -- 172.21.15.132:0/2279600909 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c004b50 con 0x55b4439fdf40 2024-04-23T18:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 -- 172.21.15.132:0/2279600909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 msgr2=0x55b443a07ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/2279600909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a07ca0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb23c005d80 tx=0x7fb23c009ae0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 -- 172.21.15.132:0/2279600909 shutdown_connections 2024-04-23T18:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/2279600909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 0x55b443a10800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/2279600909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a07ca0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/2279600909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b4439f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 -- 172.21.15.132:0/2279600909 >> 172.21.15.132:0/2279600909 conn(0x55b4439f9b30 msgr2=0x55b443a132e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 -- 172.21.15.132:0/2279600909 shutdown_connections 2024-04-23T18:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 -- 172.21.15.132:0/2279600909 wait complete. 2024-04-23T18:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 -- start start 2024-04-23T18:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 Processor -- start 2024-04-23T18:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b443a0e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a0e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 0x55b443a0ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541af040 con 0x55b4439fdf40 2024-04-23T18:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541ae970 con 0x55b4439fd620 2024-04-23T18:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541abbb0 con 0x55b443a08800 2024-04-23T18:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb25c1e2740 10 client.0 _ll_register_callbacks cb 0x7fb23c0118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb2527fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a0e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb2527fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a0e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56150/0 (socket says 172.21.15.132:56150) 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb2527fc640 1 -- 172.21.15.132:0/197349567 learned_addr learned my addr 172.21.15.132:0/197349567 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.067+0000 7fb252ffd640 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b443a0e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb2537fe640 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 0x55b443a0ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb2527fc640 1 -- 172.21.15.132:0/197349567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 msgr2=0x55b443a0ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb2527fc640 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 0x55b443a0ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb2527fc640 1 -- 172.21.15.132:0/197349567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 msgr2=0x55b443a0e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb2527fc640 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b443a0e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb2527fc640 1 -- 172.21.15.132:0/197349567 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b443a7dc80 con 0x55b4439fdf40 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb252ffd640 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b443a0e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb2527fc640 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a0e9e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb23c005910 tx=0x7fb23c0049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2810096fc977751f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb23a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb23a7fc640 1 -- 172.21.15.132:0/197349567 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c039030 con 0x55b4439fdf40 2024-04-23T18:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb23a7fc640 1 -- 172.21.15.132:0/197349567 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fb23c0092a0 con 0x55b4439fdf40 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb25c1e2740 1 -- 172.21.15.132:0/197349567 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb23c01bad0 con 0x55b4439fdf40 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb23a7fc640 1 -- 172.21.15.132:0/197349567 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb23c043050 con 0x55b4439fdf40 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb25c1e2740 10 client.15816 fetch_fsmap learned FSMap version 26 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb25c1e2740 1 -- 172.21.15.132:0/197349567 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b443a7ef00 con 0x55b4439fdf40 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb23a7fc640 1 -- 172.21.15.132:0/197349567 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb23c01bad0 con 0x55b4439fdf40 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb23a7fc640 1 -- 172.21.15.132:0/197349567 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb23c004020 con 0x55b4439fdf40 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb25c1e2740 10 client.15816 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb25c1e2740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb25c1e2740 20 client.15816 populate_metadata read hostname 'smithi132' 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b443a7f580 0x55b443a819a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb25c1e2740 4 client.15816 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.068+0000 7fb25c1e2740 1 -- 172.21.15.132:0/197349567 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b44396a200 con 0x55b443a7f580 2024-04-23T18:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.069+0000 7fb252ffd640 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b443a7f580 0x55b443a819a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.069+0000 7fb23a7fc640 10 client.15816 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.069+0000 7fb252ffd640 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b443a7f580 0x55b443a819a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8134a32448b4e47b server_cookie=1663ef839c73df21 in_seq=0 out_seq=0 2024-04-23T18:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb23a7fc640 1 -- 172.21.15.132:0/197349567 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b44396a200 con 0x55b443a7f580 2024-04-23T18:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb23a7fc640 10 client.15816 handle_command_reply: tid=0 2024-04-23T18:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb25c1e2740 1 client.15816 shutdown 2024-04-23T18:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb25c1e2740 1 -- 172.21.15.132:0/197349567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 msgr2=0x55b443a0e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a0e9e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb23c005910 tx=0x7fb23c0049f0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb25c1e2740 1 -- 172.21.15.132:0/197349567 shutdown_connections 2024-04-23T18:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55b443a7f580 0x55b443a819a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b443a08800 0x55b443a0ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4439fdf40 0x55b443a0e9e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb23a7fc640 0 client.15816 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb25c1e2740 1 --2- 172.21.15.132:0/197349567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b4439fd620 0x55b443a0e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb25c1e2740 1 -- 172.21.15.132:0/197349567 >> 172.21.15.132:0/197349567 conn(0x55b4439f9b30 msgr2=0x55b4439f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb25c1e2740 1 -- 172.21.15.132:0/197349567 shutdown_connections 2024-04-23T18:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb25c1e2740 1 -- 172.21.15.132:0/197349567 wait complete. 2024-04-23T18:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.075+0000 7fb25c1e2740 20 client.15816 trim_cache size 0 max 16384 2024-04-23T18:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-04-23T18:36:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-04-23T18:36:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-04-23T18:36:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-04-23T18:36:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.076+0000 7fb2594d0640 1 -- 172.21.15.132:0/612670475 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fb2400a3ee0 msgr2=0x7fb2400a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.076+0000 7fb2594d0640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fb2400a3ee0 0x7fb2400a63a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb24c006010 tx=0x7fb24c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.076+0000 7fb2594d0640 1 -- 172.21.15.132:0/612670475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2541006c0 msgr2=0x7fb254198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.076+0000 7fb2594d0640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2541006c0 0x7fb254198710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb24400cce0 tx=0x7fb24400a620 comp rx=0 tx=0).stop 2024-04-23T18:36:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.076+0000 7fb2594d0640 1 -- 172.21.15.132:0/612670475 shutdown_connections 2024-04-23T18:36:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.076+0000 7fb2594d0640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fb2400a3ee0 0x7fb2400a63a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.076+0000 7fb2594d0640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25419cf80 0x7fb25419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.076+0000 7fb2594d0640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb254107330 0x7fb254198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.076+0000 7fb2594d0640 1 --2- 172.21.15.132:0/612670475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2541006c0 0x7fb254198710 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.076+0000 7fb2594d0640 1 -- 172.21.15.132:0/612670475 >> 172.21.15.132:0/612670475 conn(0x7fb254077390 msgr2=0x7fb2540fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.076+0000 7fb2594d0640 1 -- 172.21.15.132:0/612670475 shutdown_connections 2024-04-23T18:36:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:36:58.076+0000 7fb2594d0640 1 -- 172.21.15.132:0/612670475 wait complete. 2024-04-23T18:36:59.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:36:58 smithi148 ceph-mon[32311]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:36:59.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:36:58 smithi157 ceph-mon[32580]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:36:59.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:36:58 smithi132 ceph-mon[29178]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:37:01.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:00 smithi148 ceph-mon[32311]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-23T18:37:01.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:00 smithi157 ceph-mon[32580]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-23T18:37:01.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:00 smithi132 ceph-mon[29178]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-23T18:37:01.111 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:37:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:37:00.625+0000 7fa6f270d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:37:01.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:37:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:37:00.625+0000 7fa6f270d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:37:01.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:37:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:37:00.625+0000 7fa6f270d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:37:01.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:37:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:37:00.625+0000 7fa6f270d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:37:01.112 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:37:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:37:00.625+0000 7fa6f270d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:37:03.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:02 smithi148 ceph-mon[32311]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-23T18:37:03.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:02 smithi148 ceph-mon[32311]: mgrmap e25: x(active, since 58s), standbys: y, z 2024-04-23T18:37:03.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:02 smithi157 ceph-mon[32580]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-23T18:37:03.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:02 smithi157 ceph-mon[32580]: mgrmap e25: x(active, since 58s), standbys: y, z 2024-04-23T18:37:03.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:02 smithi132 ceph-mon[29178]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-23T18:37:03.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:02 smithi132 ceph-mon[29178]: mgrmap e25: x(active, since 58s), standbys: y, z 2024-04-23T18:37:05.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:04 smithi148 ceph-mon[32311]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-23T18:37:05.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:04 smithi157 ceph-mon[32580]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-23T18:37:05.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:04 smithi132 ceph-mon[29178]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-23T18:37:06.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:05 smithi148 ceph-mon[32311]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-23T18:37:06.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:05 smithi157 ceph-mon[32580]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-23T18:37:06.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:05 smithi132 ceph-mon[29178]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-23T18:37:08.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:07 smithi132 ceph-mon[29178]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-23T18:37:08.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:07 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:37:08.130 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:37:08.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.240+0000 7f111c867640 1 -- 172.21.15.132:0/2290243888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11181010a0 msgr2=0x7f1118101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.240+0000 7f111c867640 1 --2- 172.21.15.132:0/2290243888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11181010a0 0x7f1118101480 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1108009900 tx=0x7f110802a410 comp rx=0 tx=0).stop 2024-04-23T18:37:08.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.240+0000 7f111c867640 1 -- 172.21.15.132:0/2290243888 shutdown_connections 2024-04-23T18:37:08.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.240+0000 7f111c867640 1 --2- 172.21.15.132:0/2290243888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1118101a50 0x7f1118105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.240+0000 7f111c867640 1 --2- 172.21.15.132:0/2290243888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1118106170 0x7f111810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.240+0000 7f111c867640 1 --2- 172.21.15.132:0/2290243888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11181010a0 0x7f1118101480 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.240+0000 7f111c867640 1 -- 172.21.15.132:0/2290243888 >> 172.21.15.132:0/2290243888 conn(0x7f1118077f50 msgr2=0x7f1118078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.240+0000 7f111c867640 1 -- 172.21.15.132:0/2290243888 shutdown_connections 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.240+0000 7f111c867640 1 -- 172.21.15.132:0/2290243888 wait complete. 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.240+0000 7f111c867640 1 Processor -- start 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.240+0000 7f111c867640 1 -- start start 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f111c867640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11181010a0 0x7f1118198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f111c867640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1118106170 0x7f1118198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f111c867640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f111819d020 0x7f111819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f111c867640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f11181110b0 con 0x7f1118106170 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f111c867640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1118110f30 con 0x7f11181010a0 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f111c867640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1118111230 con 0x7f111819d020 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f110ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1118106170 0x7f1118198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f110ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1118106170 0x7f1118198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60112/0 (socket says 172.21.15.132:60112) 2024-04-23T18:37:08.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f1116d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f111819d020 0x7f111819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f110ffff640 1 -- 172.21.15.132:0/733134546 learned_addr learned my addr 172.21.15.132:0/733134546 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:08.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f111659c640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11181010a0 0x7f1118198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f110ffff640 1 -- 172.21.15.132:0/733134546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f111819d020 msgr2=0x7f111819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f110ffff640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f111819d020 0x7f111819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f110ffff640 1 -- 172.21.15.132:0/733134546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11181010a0 msgr2=0x7f1118198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f110ffff640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11181010a0 0x7f1118198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f110ffff640 1 -- 172.21.15.132:0/733134546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111819db10 con 0x7f1118106170 2024-04-23T18:37:08.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f1116d9d640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f111819d020 0x7f111819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:37:08.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.241+0000 7f110ffff640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1118106170 0x7f1118198ca0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1100005fd0 tx=0x7f110000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:08.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.242+0000 7f110f7fe640 1 -- 172.21.15.132:0/733134546 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100017070 con 0x7f1118106170 2024-04-23T18:37:08.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.242+0000 7f111c867640 1 -- 172.21.15.132:0/733134546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111819de00 con 0x7f1118106170 2024-04-23T18:37:08.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.242+0000 7f110f7fe640 1 -- 172.21.15.132:0/733134546 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1100009c30 con 0x7f1118106170 2024-04-23T18:37:08.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.242+0000 7f110f7fe640 1 -- 172.21.15.132:0/733134546 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100003d30 con 0x7f1118106170 2024-04-23T18:37:08.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.242+0000 7f111c867640 1 -- 172.21.15.132:0/733134546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11181a5640 con 0x7f1118106170 2024-04-23T18:37:08.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.243+0000 7f111c867640 1 -- 172.21.15.132:0/733134546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1118068d70 con 0x7f1118106170 2024-04-23T18:37:08.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.244+0000 7f110f7fe640 1 -- 172.21.15.132:0/733134546 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f1100003ed0 con 0x7f1118106170 2024-04-23T18:37:08.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.244+0000 7f110f7fe640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f11040a3ed0 0x7f11040a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.245+0000 7f110f7fe640 1 -- 172.21.15.132:0/733134546 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f11000d06f0 con 0x7f1118106170 2024-04-23T18:37:08.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.245+0000 7f111659c640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f11040a3ed0 0x7f11040a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.245+0000 7f111659c640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f11040a3ed0 0x7f11040a6390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1108031440 tx=0x7f1108005b50 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:08.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.249+0000 7f110f7fe640 1 -- 172.21.15.132:0/733134546 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1100020030 con 0x7f1118106170 2024-04-23T18:37:08.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:07 smithi148 ceph-mon[32311]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-23T18:37:08.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:07 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:37:08.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:07 smithi157 ceph-mon[32580]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-23T18:37:08.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:07 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:37:08.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.473+0000 7f111c867640 1 -- 172.21.15.132:0/733134546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1118101a80 con 0x7f1118106170 2024-04-23T18:37:08.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.474+0000 7f110f7fe640 1 -- 172.21.15.132:0/733134546 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11946 (secure 0 0 0) 0x7f1100009de0 con 0x7f1118106170 2024-04-23T18:37:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:37:08.466 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:36:01.455959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24460,24425,24394,24491,24515]},"id":1}]} 2024-04-23T18:37:08.466 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 26 2024-04-23T18:37:08.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.476+0000 7f111c867640 1 -- 172.21.15.132:0/733134546 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f11040a3ed0 msgr2=0x7f11040a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.476+0000 7f111c867640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f11040a3ed0 0x7f11040a6390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1108031440 tx=0x7f1108005b50 comp rx=0 tx=0).stop 2024-04-23T18:37:08.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.476+0000 7f111c867640 1 -- 172.21.15.132:0/733134546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1118106170 msgr2=0x7f1118198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.476+0000 7f111c867640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1118106170 0x7f1118198ca0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1100005fd0 tx=0x7f110000dde0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.476+0000 7f111c867640 1 -- 172.21.15.132:0/733134546 shutdown_connections 2024-04-23T18:37:08.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.476+0000 7f111c867640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f11040a3ed0 0x7f11040a6390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.476+0000 7f111c867640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f111819d020 0x7f111819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.476+0000 7f111c867640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1118106170 0x7f1118198ca0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.476+0000 7f111c867640 1 --2- 172.21.15.132:0/733134546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11181010a0 0x7f1118198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.476+0000 7f111c867640 1 -- 172.21.15.132:0/733134546 >> 172.21.15.132:0/733134546 conn(0x7f1118077f50 msgr2=0x7f1118104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:08.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.476+0000 7f111c867640 1 -- 172.21.15.132:0/733134546 shutdown_connections 2024-04-23T18:37:08.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.476+0000 7f111c867640 1 -- 172.21.15.132:0/733134546 wait complete. 2024-04-23T18:37:08.480 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24515 quiesce db --set-id 320f00aa --release --await' 2024-04-23T18:37:08.480 DEBUG:teuthology.orchestra.run.smithi132:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24515 quiesce db --set-id 320f00aa --release --await 2024-04-23T18:37:08.591 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 -- 172.21.15.132:0/3579808498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25c1031d0 msgr2=0x7ff25c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.591 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 --2- 172.21.15.132:0/3579808498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25c1031d0 0x7ff25c1035b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff250009900 tx=0x7ff25002a510 comp rx=0 tx=0).stop 2024-04-23T18:37:08.591 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 -- 172.21.15.132:0/3579808498 shutdown_connections 2024-04-23T18:37:08.591 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 --2- 172.21.15.132:0/3579808498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c10c510 0x7ff25c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.591 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 --2- 172.21.15.132:0/3579808498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c103af0 0x7ff25c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.591 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 --2- 172.21.15.132:0/3579808498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25c1031d0 0x7ff25c1035b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.591 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 -- 172.21.15.132:0/3579808498 >> 172.21.15.132:0/3579808498 conn(0x7ff25c077f50 msgr2=0x7ff25c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:08.591 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 -- 172.21.15.132:0/3579808498 shutdown_connections 2024-04-23T18:37:08.591 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 -- 172.21.15.132:0/3579808498 wait complete. 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 Processor -- start 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 -- start start 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25c1031d0 0x7ff25c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c103af0 0x7ff25c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c10c510 0x7ff25c19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c112180 con 0x7ff25c1031d0 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c112000 con 0x7ff25c10c510 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff262ff7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c112300 con 0x7ff25c103af0 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.596+0000 7ff260d93640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25c1031d0 0x7ff25c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff260d93640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25c1031d0 0x7ff25c198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60128/0 (socket says 172.21.15.132:60128) 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff260d93640 1 -- 172.21.15.132:0/2563511646 learned_addr learned my addr 172.21.15.132:0/2563511646 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff25bfff640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c103af0 0x7ff25c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff261594640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c10c510 0x7ff25c19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff260d93640 1 -- 172.21.15.132:0/2563511646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c103af0 msgr2=0x7ff25c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff260d93640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c103af0 0x7ff25c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff260d93640 1 -- 172.21.15.132:0/2563511646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c10c510 msgr2=0x7ff25c19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff260d93640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c10c510 0x7ff25c19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff260d93640 1 -- 172.21.15.132:0/2563511646 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c19d6c0 con 0x7ff25c1031d0 2024-04-23T18:37:08.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff260d93640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25c1031d0 0x7ff25c198720 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff250005940 tx=0x7ff2500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:08.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff259ffb640 1 -- 172.21.15.132:0/2563511646 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25003f030 con 0x7ff25c1031d0 2024-04-23T18:37:08.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff259ffb640 1 -- 172.21.15.132:0/2563511646 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff250003cb0 con 0x7ff25c1031d0 2024-04-23T18:37:08.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff262ff7640 1 -- 172.21.15.132:0/2563511646 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25c19d950 con 0x7ff25c1031d0 2024-04-23T18:37:08.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff259ffb640 1 -- 172.21.15.132:0/2563511646 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250035070 con 0x7ff25c1031d0 2024-04-23T18:37:08.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.597+0000 7ff262ff7640 1 -- 172.21.15.132:0/2563511646 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25c1a5240 con 0x7ff25c1031d0 2024-04-23T18:37:08.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff259ffb640 1 -- 172.21.15.132:0/2563511646 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7ff250042030 con 0x7ff25c1031d0 2024-04-23T18:37:08.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff265de5740 1 Processor -- start 2024-04-23T18:37:08.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff265de5740 1 -- start start 2024-04-23T18:37:08.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff265de5740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 0x55799b5d6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff265de5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6cb0 0x55799b5e5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff265de5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5e5890 0x55799b5e7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff265de5740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55799b53b6d0 con 0x55799b5d6390 2024-04-23T18:37:08.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff265de5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55799b4d81a0 con 0x55799b5e5890 2024-04-23T18:37:08.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff265de5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55799b537690 con 0x55799b5d6cb0 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff261594640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 0x55799b5d6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff261594640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 0x55799b5d6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60132/0 (socket says 172.21.15.132:60132) 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff261594640 1 -- 172.21.15.132:0/2185176632 learned_addr learned my addr 172.21.15.132:0/2185176632 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff25bfff640 1 --2- 172.21.15.132:0/2185176632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5e5890 0x55799b5e7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff259ffb640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff24c0a3ed0 0x7ff24c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff260d93640 1 --2- 172.21.15.132:0/2185176632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6cb0 0x55799b5e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.600+0000 7ff261594640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff24c0a3ed0 0x7ff24c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.601+0000 7ff261594640 1 -- 172.21.15.132:0/2185176632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6cb0 msgr2=0x55799b5e5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.601+0000 7ff261594640 1 --2- 172.21.15.132:0/2185176632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6cb0 0x55799b5e5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.601+0000 7ff261594640 1 -- 172.21.15.132:0/2185176632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5e5890 msgr2=0x55799b5e7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.601+0000 7ff261594640 1 --2- 172.21.15.132:0/2185176632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5e5890 0x55799b5e7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.601+0000 7ff261594640 1 -- 172.21.15.132:0/2185176632 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799b4e25b0 con 0x55799b5d6390 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.601+0000 7ff25bfff640 1 --2- 172.21.15.132:0/2185176632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5e5890 0x55799b5e7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:37:08.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.601+0000 7ff259ffb640 1 -- 172.21.15.132:0/2563511646 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7ff2500f1440 con 0x7ff25c1031d0 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.601+0000 7ff261594640 1 --2- 172.21.15.132:0/2185176632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 0x55799b5d6770 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff254012890 tx=0x7ff254012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835e956f5fd4239 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.601+0000 7ff246ffd640 1 -- 172.21.15.132:0/2185176632 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254005020 con 0x55799b5d6390 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.601+0000 7ff261594640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff24c0a3ed0 0x7ff24c0a6390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff25401f000 tx=0x7ff25401e000 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.601+0000 7ff246ffd640 1 -- 172.21.15.132:0/2185176632 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff254005730 con 0x55799b5d6390 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.601+0000 7ff246ffd640 1 -- 172.21.15.132:0/2185176632 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25401c750 con 0x55799b5d6390 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.602+0000 7ff265de5740 1 -- 172.21.15.132:0/2185176632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 msgr2=0x55799b5d6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.602+0000 7ff265de5740 1 --2- 172.21.15.132:0/2185176632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 0x55799b5d6770 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff254012890 tx=0x7ff254012d50 comp rx=0 tx=0).stop 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.602+0000 7ff265de5740 1 -- 172.21.15.132:0/2185176632 shutdown_connections 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.602+0000 7ff265de5740 1 --2- 172.21.15.132:0/2185176632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6cb0 0x55799b5e5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.602+0000 7ff265de5740 1 --2- 172.21.15.132:0/2185176632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5e5890 0x55799b5e7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.602+0000 7ff265de5740 1 --2- 172.21.15.132:0/2185176632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 0x55799b5d6770 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.602+0000 7ff265de5740 1 -- 172.21.15.132:0/2185176632 >> 172.21.15.132:0/2185176632 conn(0x55799b5d27b0 msgr2=0x55799b5d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.602+0000 7ff265de5740 1 -- 172.21.15.132:0/2185176632 shutdown_connections 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.602+0000 7ff265de5740 1 -- 172.21.15.132:0/2185176632 wait complete. 2024-04-23T18:37:08.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.602+0000 7ff265de5740 1 -- start start 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.602+0000 7ff265de5740 1 Processor -- start 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff265de5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6390 0x55799b5e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff265de5740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6cb0 0x55799b5e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff265de5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5e5890 0x55799b5e2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff265de5740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55799b5e86f0 con 0x55799b5d6cb0 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff265de5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55799b4d5b20 con 0x55799b5e5890 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff265de5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55799b5e8870 con 0x55799b5d6390 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff265de5740 10 client.0 _ll_register_callbacks cb 0x55799b4d5530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff25bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6cb0 0x55799b5e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff25bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6cb0 0x55799b5e25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60136/0 (socket says 172.21.15.132:60136) 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff25bfff640 1 -- 172.21.15.132:0/626345632 learned_addr learned my addr 172.21.15.132:0/626345632 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff261594640 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5e5890 0x55799b5e2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff25bfff640 1 -- 172.21.15.132:0/626345632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6390 msgr2=0x55799b5e52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff260d93640 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6390 0x55799b5e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff25bfff640 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6390 0x55799b5e52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff25bfff640 1 -- 172.21.15.132:0/626345632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5e5890 msgr2=0x55799b5e2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff25bfff640 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5e5890 0x55799b5e2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff25bfff640 1 -- 172.21.15.132:0/626345632 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799b24d920 con 0x55799b5d6cb0 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff261594640 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5e5890 0x55799b5e2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff25bfff640 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6cb0 0x55799b5e25b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff24800bec0 tx=0x7ff248012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523c17768561c6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff23bfff640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff23bfff640 1 -- 172.21.15.132:0/626345632 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248010960 con 0x55799b5d6cb0 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff23bfff640 1 -- 172.21.15.132:0/626345632 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7ff248010b00 con 0x55799b5d6cb0 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff265de5740 1 -- 172.21.15.132:0/626345632 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff254005730 con 0x55799b5d6cb0 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.603+0000 7ff23bfff640 1 -- 172.21.15.132:0/626345632 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff248010d00 con 0x55799b5d6cb0 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.604+0000 7ff265de5740 10 client.15840 fetch_fsmap learned FSMap version 26 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.604+0000 7ff265de5740 1 -- 172.21.15.132:0/626345632 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55799b319d30 con 0x55799b5d6cb0 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.604+0000 7ff23bfff640 1 -- 172.21.15.132:0/626345632 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff254005730 con 0x55799b5d6cb0 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.604+0000 7ff23bfff640 1 -- 172.21.15.132:0/626345632 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff248019070 con 0x55799b5d6cb0 2024-04-23T18:37:08.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.604+0000 7ff265de5740 10 client.15840 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.604+0000 7ff265de5740 10 client.15840 resolve_mds: validated gid 24515 aka daemon mds.b 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.604+0000 7ff265de5740 20 client.15840 populate_metadata read hostname 'smithi132' 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.604+0000 7ff265de5740 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55799b5f5b40 0x55799b5f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.604+0000 7ff265de5740 4 client.15840 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.604+0000 7ff265de5740 1 -- 172.21.15.132:0/626345632 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55799b4f9500 con 0x55799b5f5b40 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.604+0000 7ff260d93640 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55799b5f5b40 0x55799b5f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff23bfff640 10 client.15840 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff260d93640 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55799b5f5b40 0x55799b5f7f80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8975a70af2d506 server_cookie=d8a5bdd8701218cc in_seq=0 out_seq=0 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff23bfff640 1 -- 172.21.15.132:0/626345632 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55799b4f9500 con 0x55799b5f5b40 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff23bfff640 10 client.15840 handle_command_reply: tid=0 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff265de5740 1 client.15840 shutdown 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff265de5740 1 -- 172.21.15.132:0/626345632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6cb0 msgr2=0x55799b5e25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff265de5740 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6cb0 0x55799b5e25b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff24800bec0 tx=0x7ff248012820 comp rx=0 tx=0).stop 2024-04-23T18:37:08.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff265de5740 1 -- 172.21.15.132:0/626345632 shutdown_connections 2024-04-23T18:37:08.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff265de5740 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55799b5f5b40 0x55799b5f7f80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff265de5740 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5e5890 0x55799b5e2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff265de5740 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6cb0 0x55799b5e25b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff265de5740 1 --2- 172.21.15.132:0/626345632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6390 0x55799b5e52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff23bfff640 0 client.15840 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:08.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff265de5740 1 -- 172.21.15.132:0/626345632 >> 172.21.15.132:0/626345632 conn(0x55799b5d27b0 msgr2=0x55799b5d85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:08.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff265de5740 1 -- 172.21.15.132:0/626345632 shutdown_connections 2024-04-23T18:37:08.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.605+0000 7ff265de5740 1 -- 172.21.15.132:0/626345632 wait complete. 2024-04-23T18:37:08.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.606+0000 7ff265de5740 20 client.15840 trim_cache size 0 max 16384 2024-04-23T18:37:08.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff265de5740 1 Processor -- start 2024-04-23T18:37:08.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff265de5740 1 -- start start 2024-04-23T18:37:08.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff265de5740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 0x55799b5d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff265de5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6cb0 0x55799b5e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff265de5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5e5890 0x55799b5e5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff265de5740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c1ab9e0 con 0x55799b5d6390 2024-04-23T18:37:08.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff265de5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c1ad520 con 0x55799b5d6cb0 2024-04-23T18:37:08.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff265de5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c1ac0b0 con 0x55799b5e5890 2024-04-23T18:37:08.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff260d93640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 0x55799b5d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff25bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6cb0 0x55799b5e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff25bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6cb0 0x55799b5e4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.132:34684/0 (socket says 172.21.15.132:34684) 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff260d93640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 0x55799b5d1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60144/0 (socket says 172.21.15.132:60144) 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff25bfff640 1 -- 172.21.15.132:0/3257214074 learned_addr learned my addr 172.21.15.132:0/3257214074 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff261594640 1 --2- 172.21.15.132:0/3257214074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5e5890 0x55799b5e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff25bfff640 1 -- 172.21.15.132:0/3257214074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5e5890 msgr2=0x55799b5e5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff25bfff640 1 --2- 172.21.15.132:0/3257214074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5e5890 0x55799b5e5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff25bfff640 1 -- 172.21.15.132:0/3257214074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 msgr2=0x55799b5d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff25bfff640 1 --2- 172.21.15.132:0/3257214074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 0x55799b5d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.623+0000 7ff25bfff640 1 -- 172.21.15.132:0/3257214074 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799b5d6ae0 con 0x55799b5d6cb0 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.624+0000 7ff25bfff640 1 --2- 172.21.15.132:0/3257214074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6cb0 0x55799b5e4df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff248023510 tx=0x7ff248012dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26108bc0784c917d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.624+0000 7ff2467fc640 1 -- 172.21.15.132:0/3257214074 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248010a20 con 0x55799b5d6cb0 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.624+0000 7ff2467fc640 1 -- 172.21.15.132:0/3257214074 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2480054e0 con 0x55799b5d6cb0 2024-04-23T18:37:08.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.624+0000 7ff2467fc640 1 -- 172.21.15.132:0/3257214074 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248023de0 con 0x55799b5d6cb0 2024-04-23T18:37:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.624+0000 7ff265de5740 1 -- 172.21.15.132:0/3257214074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6cb0 msgr2=0x55799b5e4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.624+0000 7ff265de5740 1 --2- 172.21.15.132:0/3257214074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6cb0 0x55799b5e4df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff248023510 tx=0x7ff248012dd0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 -- 172.21.15.132:0/3257214074 shutdown_connections 2024-04-23T18:37:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 --2- 172.21.15.132:0/3257214074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5e5890 0x55799b5e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 --2- 172.21.15.132:0/3257214074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6cb0 0x55799b5e4df0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 --2- 172.21.15.132:0/3257214074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5d6390 0x55799b5d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 -- 172.21.15.132:0/3257214074 >> 172.21.15.132:0/3257214074 conn(0x55799b5d27b0 msgr2=0x55799b5d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 -- 172.21.15.132:0/3257214074 shutdown_connections 2024-04-23T18:37:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 -- 172.21.15.132:0/3257214074 wait complete. 2024-04-23T18:37:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 -- start start 2024-04-23T18:37:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 Processor -- start 2024-04-23T18:37:08.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6390 0x55799b5edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6cb0 0x55799b5e8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5e5890 0x55799b5e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c1ace50 con 0x55799b5e5890 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c1ac780 con 0x55799b5d6390 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c1a99c0 con 0x55799b5d6cb0 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff265de5740 10 client.0 _ll_register_callbacks cb 0x7ff24800bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff261594640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5e5890 0x55799b5e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.625+0000 7ff25bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6cb0 0x55799b5e8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff25bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6cb0 0x55799b5e8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.132:44362/0 (socket says 172.21.15.132:44362) 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff25bfff640 1 -- 172.21.15.132:0/3765143666 learned_addr learned my addr 172.21.15.132:0/3765143666 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff260d93640 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6390 0x55799b5edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff25bfff640 1 -- 172.21.15.132:0/3765143666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6390 msgr2=0x55799b5edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff25bfff640 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6390 0x55799b5edfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff25bfff640 1 -- 172.21.15.132:0/3765143666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5e5890 msgr2=0x55799b5e90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:08.622 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff25bfff640 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5e5890 0x55799b5e90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff25bfff640 1 -- 172.21.15.132:0/3765143666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799b5e98d0 con 0x55799b5d6cb0 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff260d93640 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6390 0x55799b5edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff261594640 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5e5890 0x55799b5e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff25bfff640 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6cb0 0x55799b5e8ac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff24800c010 tx=0x7ff248023e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cce9e8feedfe19a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff23bfff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff23bfff640 1 -- 172.21.15.132:0/3765143666 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248010a30 con 0x55799b5d6cb0 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff265de5740 1 -- 172.21.15.132:0/3765143666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2480054e0 con 0x55799b5d6cb0 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff23bfff640 1 -- 172.21.15.132:0/3765143666 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7ff248031040 con 0x55799b5d6cb0 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff23bfff640 1 -- 172.21.15.132:0/3765143666 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff248035480 con 0x55799b5d6cb0 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff265de5740 10 client.25258 fetch_fsmap learned FSMap version 26 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.626+0000 7ff265de5740 1 -- 172.21.15.132:0/3765143666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55799b65d200 con 0x55799b5d6cb0 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.627+0000 7ff23bfff640 1 -- 172.21.15.132:0/3765143666 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2480054e0 con 0x55799b5d6cb0 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.627+0000 7ff23bfff640 1 -- 172.21.15.132:0/3765143666 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff24802a070 con 0x55799b5d6cb0 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.627+0000 7ff265de5740 10 client.25258 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:37:08.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.627+0000 7ff265de5740 10 client.25258 resolve_mds: validated gid 24515 aka daemon mds.b 2024-04-23T18:37:08.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.627+0000 7ff265de5740 20 client.25258 populate_metadata read hostname 'smithi132' 2024-04-23T18:37:08.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.627+0000 7ff265de5740 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55799b65d760 0x55799b65fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:08.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.627+0000 7ff265de5740 4 client.25258 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "320f00aa", "release": true, "await": true}] 2024-04-23T18:37:08.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.627+0000 7ff265de5740 1 -- 172.21.15.132:0/3765143666 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "quiesce db", "set_id": "320f00aa", "release": true, "await": true}) -- 0x55799b5f22a0 con 0x55799b65d760 2024-04-23T18:37:08.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.627+0000 7ff260d93640 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55799b65d760 0x55799b65fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.627+0000 7ff23bfff640 10 client.25258 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:08.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.627+0000 7ff260d93640 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55799b65d760 0x55799b65fb80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30cbd2026762b193 server_cookie=ed1872a50bbf5808 in_seq=0 out_seq=0 2024-04-23T18:37:08.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff23bfff640 1 -- 172.21.15.132:0/3765143666 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55799b5f22a0 con 0x55799b65d760 2024-04-23T18:37:08.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff23bfff640 10 client.25258 handle_command_reply: tid=0 2024-04-23T18:37:08.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff265de5740 1 client.25258 shutdown 2024-04-23T18:37:08.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff265de5740 1 -- 172.21.15.132:0/3765143666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6cb0 msgr2=0x55799b5e8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff265de5740 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6cb0 0x55799b5e8ac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff24800c010 tx=0x7ff248023e90 comp rx=0 tx=0).stop 2024-04-23T18:37:08.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff265de5740 1 -- 172.21.15.132:0/3765143666 shutdown_connections 2024-04-23T18:37:08.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff265de5740 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x55799b65d760 0x55799b65fb80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff265de5740 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55799b5d6cb0 0x55799b5e8ac0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff23bfff640 0 client.25258 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:08.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff265de5740 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55799b5e5890 0x55799b5e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff265de5740 1 --2- 172.21.15.132:0/3765143666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55799b5d6390 0x55799b5edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff265de5740 1 -- 172.21.15.132:0/3765143666 >> 172.21.15.132:0/3765143666 conn(0x55799b5d27b0 msgr2=0x55799b5d8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:08.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff265de5740 1 -- 172.21.15.132:0/3765143666 shutdown_connections 2024-04-23T18:37:08.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff265de5740 1 -- 172.21.15.132:0/3765143666 wait complete. 2024-04-23T18:37:08.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.631+0000 7ff265de5740 20 client.25258 trim_cache size 0 max 16384 2024-04-23T18:37:08.626 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:37:08.626 INFO:teuthology.orchestra.run.smithi132.stdout: "epoch": 26, 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "leader": 24515, 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "set_version": 8, 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "sets": { 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "320f00aa": { 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "version": 8, 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "age_ref": 0.0, 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "state": { 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RELEASED", 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "age": 0.0 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "timeout": 60.0, 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "expiration": 79.5, 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "members": { 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "file:/": { 2024-04-23T18:37:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "excluded": false, 2024-04-23T18:37:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "state": { 2024-04-23T18:37:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RELEASED", 2024-04-23T18:37:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "age": 0.0 2024-04-23T18:37:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:37:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:37:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:37:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:37:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:37:08.628 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:37:08.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.632+0000 7ff262ff7640 1 -- 172.21.15.132:0/2563511646 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff24c0a3ed0 msgr2=0x7ff24c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.632+0000 7ff262ff7640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff24c0a3ed0 0x7ff24c0a6390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff25401f000 tx=0x7ff25401e000 comp rx=0 tx=0).stop 2024-04-23T18:37:08.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.632+0000 7ff262ff7640 1 -- 172.21.15.132:0/2563511646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25c1031d0 msgr2=0x7ff25c198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.632+0000 7ff262ff7640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25c1031d0 0x7ff25c198720 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff250005940 tx=0x7ff2500042f0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.632+0000 7ff262ff7640 1 -- 172.21.15.132:0/2563511646 shutdown_connections 2024-04-23T18:37:08.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.632+0000 7ff262ff7640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff24c0a3ed0 0x7ff24c0a6390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.632+0000 7ff262ff7640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c103af0 0x7ff25c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.632+0000 7ff262ff7640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c10c510 0x7ff25c19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.632+0000 7ff262ff7640 1 --2- 172.21.15.132:0/2563511646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff25c1031d0 0x7ff25c198720 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:08.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.632+0000 7ff262ff7640 1 -- 172.21.15.132:0/2563511646 >> 172.21.15.132:0/2563511646 conn(0x7ff25c077f50 msgr2=0x7ff25c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:08.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.632+0000 7ff262ff7640 1 -- 172.21.15.132:0/2563511646 shutdown_connections 2024-04-23T18:37:08.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:08.632+0000 7ff262ff7640 1 -- 172.21.15.132:0/2563511646 wait complete. 2024-04-23T18:37:08.630 INFO:tasks.workunit.client.0.smithi132.stdout:test large (16MB) dio write 2024-04-23T18:37:08.630 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test large (16MB) dio write' 2024-04-23T18:37:08.630 INFO:tasks.workunit.client.0.smithi132.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-04-23T18:37:08.639 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '320f00aa', seconds elapsed: 20.6 2024-04-23T18:37:08.709 INFO:tasks.workunit.client.0.smithi132.stderr:1+0 records in 2024-04-23T18:37:08.709 INFO:tasks.workunit.client.0.smithi132.stderr:1+0 records out 2024-04-23T18:37:08.709 INFO:tasks.workunit.client.0.smithi132.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0794168 s, 211 MB/s 2024-04-23T18:37:08.709 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo OK 2024-04-23T18:37:08.710 INFO:tasks.workunit.client.0.smithi132.stdout:OK 2024-04-23T18:37:08.711 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-23T18:37:08.711 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-23T18:37:08.744 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-04-23T18:37:08.745 DEBUG:teuthology.orchestra.run.smithi132:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57e0caf76fdb7ab8b1358588f292d08519163844 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-04-23T18:37:09.426 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:09 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:37:09.426 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:09 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:37:09.426 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:09 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/733134546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:37:09.426 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:09 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:37:09.426 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:09 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:37:09.426 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:09 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:37:09.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:09 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:37:09.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:09 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:37:09.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:09 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/733134546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:37:09.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:09 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:37:09.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:09 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:37:09.540 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:09 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:37:09.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:09 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:37:09.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:09 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:37:09.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:09 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/733134546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:37:09.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:09 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:37:09.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:09 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:37:09.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:09 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:37:10.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:10 smithi148 ceph-mon[32311]: pgmap v36: 97 pgs: 97 active+clean; 17 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 29 op/s 2024-04-23T18:37:10.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:10 smithi157 ceph-mon[32580]: pgmap v36: 97 pgs: 97 active+clean; 17 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 29 op/s 2024-04-23T18:37:10.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:10 smithi132 ceph-mon[29178]: pgmap v36: 97 pgs: 97 active+clean; 17 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 29 op/s 2024-04-23T18:37:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.163+0000 7fadecfcd640 1 -- 172.21.15.132:0/1064478571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade8074c20 msgr2=0x7fade810e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.163+0000 7fadecfcd640 1 --2- 172.21.15.132:0/1064478571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade8074c20 0x7fade810e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009900 tx=0x7fadd802a480 comp rx=0 tx=0).stop 2024-04-23T18:37:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.164+0000 7fadecfcd640 1 -- 172.21.15.132:0/1064478571 shutdown_connections 2024-04-23T18:37:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.164+0000 7fadecfcd640 1 --2- 172.21.15.132:0/1064478571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade810e830 0x7fade8110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.164+0000 7fadecfcd640 1 --2- 172.21.15.132:0/1064478571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade8074c20 0x7fade810e2f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.164+0000 7fadecfcd640 1 --2- 172.21.15.132:0/1064478571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fade8074300 0x7fade80746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.164+0000 7fadecfcd640 1 -- 172.21.15.132:0/1064478571 >> 172.21.15.132:0/1064478571 conn(0x7fade80fda00 msgr2=0x7fade80ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.164+0000 7fadecfcd640 1 -- 172.21.15.132:0/1064478571 shutdown_connections 2024-04-23T18:37:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.164+0000 7fadecfcd640 1 -- 172.21.15.132:0/1064478571 wait complete. 2024-04-23T18:37:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.164+0000 7fadecfcd640 1 Processor -- start 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.164+0000 7fadecfcd640 1 -- start start 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fadecfcd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fade8074300 0x7fade81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fadecfcd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade8074c20 0x7fade81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fadecfcd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade810e830 0x7fade819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fadecfcd640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fade81143d0 con 0x7fade8074c20 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fadecfcd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fade8114250 con 0x7fade8074300 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fadecfcd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fade8114550 con 0x7fade810e830 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fade5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade8074c20 0x7fade81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fade5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade8074c20 0x7fade81a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34360/0 (socket says 172.21.15.132:34360) 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fade5d9b640 1 -- 172.21.15.132:0/772063771 learned_addr learned my addr 172.21.15.132:0/772063771 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fade6d9d640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade810e830 0x7fade819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fade659c640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fade8074300 0x7fade81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fade5d9b640 1 -- 172.21.15.132:0/772063771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade810e830 msgr2=0x7fade819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fade5d9b640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade810e830 0x7fade819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fade5d9b640 1 -- 172.21.15.132:0/772063771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fade8074300 msgr2=0x7fade81a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fade5d9b640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fade8074300 0x7fade81a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fade5d9b640 1 -- 172.21.15.132:0/772063771 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade819aff0 con 0x7fade8074c20 2024-04-23T18:37:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fade6d9d640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade810e830 0x7fade819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:37:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fade5d9b640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade8074c20 0x7fade81a0b60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005940 tx=0x7fadd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fadd77fe640 1 -- 172.21.15.132:0/772063771 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd803f030 con 0x7fade8074c20 2024-04-23T18:37:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fadecfcd640 1 -- 172.21.15.132:0/772063771 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade819b280 con 0x7fade8074c20 2024-04-23T18:37:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fadd77fe640 1 -- 172.21.15.132:0/772063771 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd8003aa0 con 0x7fade8074c20 2024-04-23T18:37:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fadd77fe640 1 -- 172.21.15.132:0/772063771 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8035070 con 0x7fade8074c20 2024-04-23T18:37:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.165+0000 7fadecfcd640 1 -- 172.21.15.132:0/772063771 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade81a75a0 con 0x7fade8074c20 2024-04-23T18:37:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.168+0000 7fadd77fe640 1 -- 172.21.15.132:0/772063771 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fadd8042030 con 0x7fade8074c20 2024-04-23T18:37:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.168+0000 7fadefdc4740 1 Processor -- start 2024-04-23T18:37:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.168+0000 7fadefdc4740 1 -- start start 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.168+0000 7fadefdc4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43620 0x555bb2f43a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.168+0000 7fadefdc4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43f40 0x555bb2f4e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.168+0000 7fadefdc4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f50bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.168+0000 7fadefdc4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x555bb2ea8970 con 0x555bb2f4e800 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.168+0000 7fadefdc4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555bb2e454a0 con 0x555bb2f43f40 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.168+0000 7fadefdc4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555bb2ea4930 con 0x555bb2f43620 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fadd77fe640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fadd00a3f70 0x7fadd00a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43620 0x555bb2f43a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f50bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43620 0x555bb2f43a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.132:39046/0 (socket says 172.21.15.132:39046) 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade6d9d640 1 -- 172.21.15.132:0/2539405111 learned_addr learned my addr 172.21.15.132:0/2539405111 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade659c640 1 --2- 172.21.15.132:0/2539405111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43f40 0x555bb2f4e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade6d9d640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fadd00a3f70 0x7fadd00a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fadd77fe640 1 -- 172.21.15.132:0/772063771 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fadd80f1090 con 0x7fade8074c20 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade5d9b640 1 -- 172.21.15.132:0/2539405111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43620 msgr2=0x555bb2f43a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade5d9b640 1 --2- 172.21.15.132:0/2539405111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43620 0x555bb2f43a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade5d9b640 1 -- 172.21.15.132:0/2539405111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43f40 msgr2=0x555bb2f4e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade5d9b640 1 --2- 172.21.15.132:0/2539405111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43f40 0x555bb2f4e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade5d9b640 1 -- 172.21.15.132:0/2539405111 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb2bbb210 con 0x555bb2f4e800 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade5d9b640 1 --2- 172.21.15.132:0/2539405111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f50bf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fadd80ee590 tx=0x7fadd802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6ac8dd45a9211 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fadd4ff9640 1 -- 172.21.15.132:0/2539405111 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8049070 con 0x555bb2f4e800 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fade6d9d640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fadd00a3f70 0x7fadd00a6430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fade0005c40 tx=0x7fade0004ab0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fadd4ff9640 1 -- 172.21.15.132:0/2539405111 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd80bce90 con 0x555bb2f4e800 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.169+0000 7fadd4ff9640 1 -- 172.21.15.132:0/2539405111 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80b9500 con 0x555bb2f4e800 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.170+0000 7fadefdc4740 1 -- 172.21.15.132:0/2539405111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 msgr2=0x555bb2f50bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.170+0000 7fadefdc4740 1 --2- 172.21.15.132:0/2539405111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f50bf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fadd80ee590 tx=0x7fadd802a480 comp rx=0 tx=0).stop 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.170+0000 7fadefdc4740 1 -- 172.21.15.132:0/2539405111 shutdown_connections 2024-04-23T18:37:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.170+0000 7fadefdc4740 1 --2- 172.21.15.132:0/2539405111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f50bf0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.170+0000 7fadefdc4740 1 --2- 172.21.15.132:0/2539405111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43f40 0x555bb2f4e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.170+0000 7fadefdc4740 1 --2- 172.21.15.132:0/2539405111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43620 0x555bb2f43a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.170+0000 7fadefdc4740 1 -- 172.21.15.132:0/2539405111 >> 172.21.15.132:0/2539405111 conn(0x555bb2f3fb30 msgr2=0x555bb2f3ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.170+0000 7fadefdc4740 1 -- 172.21.15.132:0/2539405111 shutdown_connections 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.170+0000 7fadefdc4740 1 -- 172.21.15.132:0/2539405111 wait complete. 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.170+0000 7fadefdc4740 1 -- start start 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fadefdc4740 1 Processor -- start 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fadefdc4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43620 0x555bb2f4e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fadefdc4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43f40 0x555bb2f4b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fadefdc4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f4b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fadefdc4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x555bb2ea8970 con 0x555bb2f4e800 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fadefdc4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555bb2e430b0 con 0x555bb2f43620 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fadefdc4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555bb2f511f0 con 0x555bb2f43f40 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fadefdc4740 10 client.0 _ll_register_callbacks cb 0x555bb2e30d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fade6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f4b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fade6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f4b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34372/0 (socket says 172.21.15.132:34372) 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fade6d9d640 1 -- 172.21.15.132:0/3799350786 learned_addr learned my addr 172.21.15.132:0/3799350786 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fade659c640 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43620 0x555bb2f4e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fade5d9b640 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43f40 0x555bb2f4b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fade6d9d640 1 -- 172.21.15.132:0/3799350786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43f40 msgr2=0x555bb2f4b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fade6d9d640 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43f40 0x555bb2f4b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fade6d9d640 1 -- 172.21.15.132:0/3799350786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43620 msgr2=0x555bb2f4e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fade6d9d640 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43620 0x555bb2f4e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fade6d9d640 1 -- 172.21.15.132:0/3799350786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb2c87620 con 0x555bb2f4e800 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fade6d9d640 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f4b9c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fade000ff90 tx=0x7fade0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a9c1dfb54804ad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.171+0000 7fadcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadcdffb640 1 -- 172.21.15.132:0/3799350786 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0004d20 con 0x555bb2f4e800 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadcdffb640 1 -- 172.21.15.132:0/3799350786 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fade001e620 con 0x555bb2f4e800 2024-04-23T18:37:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadefdc4740 1 -- 172.21.15.132:0/3799350786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd80bce90 con 0x555bb2f4e800 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadcdffb640 1 -- 172.21.15.132:0/3799350786 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fade001e820 con 0x555bb2f4e800 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadefdc4740 10 client.15858 fetch_fsmap learned FSMap version 26 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadefdc4740 1 -- 172.21.15.132:0/3799350786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555bb2f5cfa0 con 0x555bb2f4e800 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadcdffb640 1 -- 172.21.15.132:0/3799350786 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd80bce90 con 0x555bb2f4e800 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadcdffb640 1 -- 172.21.15.132:0/3799350786 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fade0027030 con 0x555bb2f4e800 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadefdc4740 10 client.15858 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadefdc4740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadefdc4740 20 client.15858 populate_metadata read hostname 'smithi132' 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555bb2f5e4d0 0x555bb2f608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadefdc4740 4 client.15858 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fadefdc4740 1 -- 172.21.15.132:0/3799350786 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bb2eb6390 con 0x555bb2f5e4d0 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.172+0000 7fade659c640 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555bb2f5e4d0 0x555bb2f608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.173+0000 7fade659c640 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555bb2f5e4d0 0x555bb2f608c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ddbdd5123d014ef server_cookie=81cbcfa2502c3b82 in_seq=0 out_seq=0 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.173+0000 7fadcdffb640 10 client.15858 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.173+0000 7fadcdffb640 1 -- 172.21.15.132:0/3799350786 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bb2eb6390 con 0x555bb2f5e4d0 2024-04-23T18:37:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.173+0000 7fadcdffb640 10 client.15858 handle_command_reply: tid=0 2024-04-23T18:37:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.173+0000 7fadefdc4740 1 client.15858 shutdown 2024-04-23T18:37:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.174+0000 7fadefdc4740 1 -- 172.21.15.132:0/3799350786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 msgr2=0x555bb2f4b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.174+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f4b9c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fade000ff90 tx=0x7fade0004a40 comp rx=0 tx=0).stop 2024-04-23T18:37:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.174+0000 7fadefdc4740 1 -- 172.21.15.132:0/3799350786 shutdown_connections 2024-04-23T18:37:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.174+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555bb2f5e4d0 0x555bb2f608c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.174+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43f40 0x555bb2f4b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.174+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f4b9c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.174+0000 7fadcdffb640 0 client.15858 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.174+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3799350786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43620 0x555bb2f4e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.174+0000 7fadefdc4740 1 -- 172.21.15.132:0/3799350786 >> 172.21.15.132:0/3799350786 conn(0x555bb2f3fb30 msgr2=0x555bb2f45860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.174+0000 7fadefdc4740 1 -- 172.21.15.132:0/3799350786 shutdown_connections 2024-04-23T18:37:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.174+0000 7fadefdc4740 1 -- 172.21.15.132:0/3799350786 wait complete. 2024-04-23T18:37:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.174+0000 7fadefdc4740 20 client.15858 trim_cache size 0 max 16384 2024-04-23T18:37:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.188+0000 7fadefdc4740 1 Processor -- start 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.188+0000 7fadefdc4740 1 -- start start 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.188+0000 7fadefdc4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f43620 0x555bb2f59f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.188+0000 7fadefdc4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43f40 0x555bb2f661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.188+0000 7fadefdc4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f4e800 0x555bb2f66720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.188+0000 7fadefdc4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fade81afe20 con 0x555bb2f43620 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.188+0000 7fadefdc4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fade81b1960 con 0x555bb2f43f40 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.188+0000 7fadefdc4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fade81b04f0 con 0x555bb2f4e800 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.188+0000 7fade659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f43620 0x555bb2f59f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.189+0000 7fade659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f43620 0x555bb2f59f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34376/0 (socket says 172.21.15.132:34376) 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.189+0000 7fade659c640 1 -- 172.21.15.132:0/3121158971 learned_addr learned my addr 172.21.15.132:0/3121158971 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.189+0000 7fade6d9d640 1 --2- 172.21.15.132:0/3121158971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f4e800 0x555bb2f66720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.189+0000 7fade659c640 1 -- 172.21.15.132:0/3121158971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f4e800 msgr2=0x555bb2f66720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.189+0000 7fade659c640 1 --2- 172.21.15.132:0/3121158971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f4e800 0x555bb2f66720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.189+0000 7fade659c640 1 -- 172.21.15.132:0/3121158971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43f40 msgr2=0x555bb2f661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.189+0000 7fade659c640 1 --2- 172.21.15.132:0/3121158971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43f40 0x555bb2f661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.189+0000 7fade659c640 1 -- 172.21.15.132:0/3121158971 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb2f60bf0 con 0x555bb2f43620 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.189+0000 7fade659c640 1 --2- 172.21.15.132:0/3121158971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f43620 0x555bb2f59f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7faddc00b8a0 tx=0x7faddc00bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e356b3b41508c95c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.189+0000 7fadcffff640 1 -- 172.21.15.132:0/3121158971 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc003ea0 con 0x555bb2f43620 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.189+0000 7fadcffff640 1 -- 172.21.15.132:0/3121158971 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faddc01d8e0 con 0x555bb2f43620 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.189+0000 7fadcffff640 1 -- 172.21.15.132:0/3121158971 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc010430 con 0x555bb2f43620 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 -- 172.21.15.132:0/3121158971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f43620 msgr2=0x555bb2f59f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3121158971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f43620 0x555bb2f59f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7faddc00b8a0 tx=0x7faddc00bef0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 -- 172.21.15.132:0/3121158971 shutdown_connections 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3121158971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f4e800 0x555bb2f66720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3121158971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43f40 0x555bb2f661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3121158971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f43620 0x555bb2f59f30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 -- 172.21.15.132:0/3121158971 >> 172.21.15.132:0/3121158971 conn(0x555bb2f3fb30 msgr2=0x555bb2f3ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 -- 172.21.15.132:0/3121158971 shutdown_connections 2024-04-23T18:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 -- 172.21.15.132:0/3121158971 wait complete. 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 -- start start 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 Processor -- start 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43620 0x555bb2f56170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43f40 0x555bb2f566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f54710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fade81b1290 con 0x555bb2f4e800 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fade81b0bc0 con 0x555bb2f43620 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fade81ade00 con 0x555bb2f43f40 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fadefdc4740 10 client.0 _ll_register_callbacks cb 0x7fade0013f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fade6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f54710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fade6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f54710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34382/0 (socket says 172.21.15.132:34382) 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.190+0000 7fade6d9d640 1 -- 172.21.15.132:0/3088820900 learned_addr learned my addr 172.21.15.132:0/3088820900 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fade5d9b640 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43f40 0x555bb2f566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fade659c640 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43620 0x555bb2f56170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fade6d9d640 1 -- 172.21.15.132:0/3088820900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43f40 msgr2=0x555bb2f566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fade6d9d640 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43f40 0x555bb2f566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fade6d9d640 1 -- 172.21.15.132:0/3088820900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43620 msgr2=0x555bb2f56170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fade6d9d640 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43620 0x555bb2f56170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fade6d9d640 1 -- 172.21.15.132:0/3088820900 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb2f54e00 con 0x555bb2f4e800 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fade6d9d640 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f54710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x555bb2f3f570 tx=0x7fade0004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f32788258440c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fadcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fadcdffb640 1 -- 172.21.15.132:0/3088820900 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade003d060 con 0x555bb2f4e800 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fadcdffb640 1 -- 172.21.15.132:0/3088820900 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fade0005210 con 0x555bb2f4e800 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fadefdc4740 1 -- 172.21.15.132:0/3088820900 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faddc01d8e0 con 0x555bb2f4e800 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fadcdffb640 1 -- 172.21.15.132:0/3088820900 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fade001e9c0 con 0x555bb2f4e800 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fadefdc4740 10 client.15864 fetch_fsmap learned FSMap version 26 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fadefdc4740 1 -- 172.21.15.132:0/3088820900 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555bb2fc5600 con 0x555bb2f4e800 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.191+0000 7fadcdffb640 1 -- 172.21.15.132:0/3088820900 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faddc01d8e0 con 0x555bb2f4e800 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.192+0000 7fadcdffb640 1 -- 172.21.15.132:0/3088820900 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fade0027040 con 0x555bb2f4e800 2024-04-23T18:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.192+0000 7fadefdc4740 10 client.15864 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:37:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.192+0000 7fadefdc4740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.192+0000 7fadefdc4740 20 client.15864 populate_metadata read hostname 'smithi132' 2024-04-23T18:37:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.192+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555bb2fc5d20 0x555bb2fc8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.192+0000 7fadefdc4740 4 client.15864 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:37:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.192+0000 7fadefdc4740 1 -- 172.21.15.132:0/3088820900 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bb2eb0200 con 0x555bb2fc5d20 2024-04-23T18:37:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.192+0000 7fade659c640 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555bb2fc5d20 0x555bb2fc8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.192+0000 7fade659c640 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555bb2fc5d20 0x555bb2fc8140 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa0b9f791a22441 server_cookie=96a7d1aa6ccf40dd in_seq=0 out_seq=0 2024-04-23T18:37:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.192+0000 7fadcdffb640 10 client.15864 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.203+0000 7fadcdffb640 1 -- 172.21.15.132:0/3088820900 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555bb2eb0200 con 0x555bb2fc5d20 2024-04-23T18:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.203+0000 7fadcdffb640 10 client.15864 handle_command_reply: tid=0 2024-04-23T18:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.203+0000 7fadefdc4740 1 client.15864 shutdown 2024-04-23T18:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.203+0000 7fadefdc4740 1 -- 172.21.15.132:0/3088820900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 msgr2=0x555bb2f54710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.203+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f54710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x555bb2f3f570 tx=0x7fade0004d50 comp rx=0 tx=0).stop 2024-04-23T18:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadefdc4740 1 -- 172.21.15.132:0/3088820900 shutdown_connections 2024-04-23T18:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555bb2fc5d20 0x555bb2fc8140 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bb2f43f40 0x555bb2f566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555bb2f4e800 0x555bb2f54710 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadcdffb640 0 client.15864 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadefdc4740 1 --2- 172.21.15.132:0/3088820900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb2f43620 0x555bb2f56170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadefdc4740 1 -- 172.21.15.132:0/3088820900 >> 172.21.15.132:0/3088820900 conn(0x555bb2f3fb30 msgr2=0x555bb2f58f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadefdc4740 1 -- 172.21.15.132:0/3088820900 shutdown_connections 2024-04-23T18:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadefdc4740 1 -- 172.21.15.132:0/3088820900 wait complete. 2024-04-23T18:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadefdc4740 20 client.15864 trim_cache size 0 max 16384 2024-04-23T18:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-04-23T18:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-04-23T18:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-04-23T18:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-04-23T18:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadecfcd640 1 -- 172.21.15.132:0/772063771 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fadd00a3f70 msgr2=0x7fadd00a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadecfcd640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fadd00a3f70 0x7fadd00a6430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fade0005c40 tx=0x7fade0004ab0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadecfcd640 1 -- 172.21.15.132:0/772063771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade8074c20 msgr2=0x7fade81a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadecfcd640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade8074c20 0x7fade81a0b60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005940 tx=0x7fadd8004270 comp rx=0 tx=0).stop 2024-04-23T18:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadecfcd640 1 -- 172.21.15.132:0/772063771 shutdown_connections 2024-04-23T18:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadecfcd640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fadd00a3f70 0x7fadd00a6430 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadecfcd640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade810e830 0x7fade819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadecfcd640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fade8074c20 0x7fade81a0b60 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadecfcd640 1 --2- 172.21.15.132:0/772063771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fade8074300 0x7fade81a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadecfcd640 1 -- 172.21.15.132:0/772063771 >> 172.21.15.132:0/772063771 conn(0x7fade80fda00 msgr2=0x7fade80fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadecfcd640 1 -- 172.21.15.132:0/772063771 shutdown_connections 2024-04-23T18:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:11.204+0000 7fadecfcd640 1 -- 172.21.15.132:0/772063771 wait complete. 2024-04-23T18:37:12.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:12 smithi148 ceph-mon[32311]: pgmap v37: 97 pgs: 97 active+clean; 17 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 26 op/s 2024-04-23T18:37:12.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:12 smithi157 ceph-mon[32580]: pgmap v37: 97 pgs: 97 active+clean; 17 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 26 op/s 2024-04-23T18:37:12.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:12 smithi132 ceph-mon[29178]: pgmap v37: 97 pgs: 97 active+clean; 17 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 26 op/s 2024-04-23T18:37:14.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:14 smithi148 ceph-mon[32311]: pgmap v38: 97 pgs: 97 active+clean; 17 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.4 MiB/s wr, 24 op/s 2024-04-23T18:37:14.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:14 smithi157 ceph-mon[32580]: pgmap v38: 97 pgs: 97 active+clean; 17 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.4 MiB/s wr, 24 op/s 2024-04-23T18:37:14.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:14 smithi132 ceph-mon[29178]: pgmap v38: 97 pgs: 97 active+clean; 17 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.4 MiB/s wr, 24 op/s 2024-04-23T18:37:16.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:16 smithi148 ceph-mon[32311]: pgmap v39: 97 pgs: 97 active+clean; 1.8 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.4 MiB/s wr, 1.64k op/s 2024-04-23T18:37:16.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:16 smithi157 ceph-mon[32580]: pgmap v39: 97 pgs: 97 active+clean; 1.8 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.4 MiB/s wr, 1.64k op/s 2024-04-23T18:37:16.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:16 smithi132 ceph-mon[29178]: pgmap v39: 97 pgs: 97 active+clean; 1.8 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.4 MiB/s wr, 1.64k op/s 2024-04-23T18:37:17.991 INFO:tasks.workunit.client.0.smithi132.stdout:.......... 2024-04-23T18:37:17.992 INFO:tasks.workunit.client.0.smithi132.stdout:do_o_direct_reads: SUCCESS 2024-04-23T18:37:17.995 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-23T18:37:17.995 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-23T18:37:18.062 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-04-23T18:37:18.062 DEBUG:teuthology.orchestra.run.smithi132:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57e0caf76fdb7ab8b1358588f292d08519163844 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-04-23T18:37:18.123 INFO:tasks.workunit.client.0.smithi132.stdout:test read from hole 2024-04-23T18:37:18.123 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test read from hole' 2024-04-23T18:37:18.124 INFO:tasks.workunit.client.0.smithi132.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-04-23T18:37:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:0+0 records in 2024-04-23T18:37:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:0+0 records out 2024-04-23T18:37:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:0 bytes copied, 3.1355e-05 s, 0.0 kB/s 2024-04-23T18:37:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-04-23T18:37:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2+0 records in 2024-04-23T18:37:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2+0 records out 2024-04-23T18:37:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00114502 s, 894 kB/s 2024-04-23T18:37:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-04-23T18:37:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:2+0 records in 2024-04-23T18:37:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:2+0 records out 2024-04-23T18:37:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 3.9336e-05 s, 26.0 MB/s 2024-04-23T18:37:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-04-23T18:37:18.128 INFO:tasks.workunit.client.0.smithi132.stdout:other thing 2024-04-23T18:37:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'other thing' 2024-04-23T18:37:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-04-23T18:37:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:1+0 records in 2024-04-23T18:37:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:1+0 records out 2024-04-23T18:37:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:500 bytes copied, 3.6948e-05 s, 13.5 MB/s 2024-04-23T18:37:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-04-23T18:37:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:0+1 records in 2024-04-23T18:37:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:0+1 records out 2024-04-23T18:37:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:500 bytes copied, 0.00552823 s, 90.4 kB/s 2024-04-23T18:37:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-04-23T18:37:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:0+1 records in 2024-04-23T18:37:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:0+1 records out 2024-04-23T18:37:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:500 bytes copied, 0.00214704 s, 233 kB/s 2024-04-23T18:37:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-04-23T18:37:18.140 INFO:tasks.workunit.client.0.smithi132.stdout:OK 2024-04-23T18:37:18.140 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo OK 2024-04-23T18:37:18.141 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-23T18:37:18.141 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-23T18:37:18.216 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-04-23T18:37:18.216 DEBUG:teuthology.orchestra.run.smithi132:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57e0caf76fdb7ab8b1358588f292d08519163844 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-04-23T18:37:18.285 INFO:tasks.workunit.client.0.smithi132.stdout:writing first 3 bytes of 10k file 2024-04-23T18:37:18.285 INFO:tasks.workunit.client.0.smithi132.stdout:reading O_DIRECT 2024-04-23T18:37:18.285 INFO:tasks.workunit.client.0.smithi132.stdout:got 10000 2024-04-23T18:37:18.285 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-23T18:37:18.286 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-23T18:37:18.345 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:18 smithi132 ceph-mon[29178]: pgmap v40: 97 pgs: 97 active+clean; 1.8 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.4 MiB/s wr, 1.61k op/s 2024-04-23T18:37:18.356 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-04-23T18:37:18.357 DEBUG:teuthology.orchestra.run.smithi132:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57e0caf76fdb7ab8b1358588f292d08519163844 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-04-23T18:37:18.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:18 smithi148 ceph-mon[32311]: pgmap v40: 97 pgs: 97 active+clean; 1.8 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.4 MiB/s wr, 1.61k op/s 2024-04-23T18:37:18.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:18 smithi157 ceph-mon[32580]: pgmap v40: 97 pgs: 97 active+clean; 1.8 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.4 MiB/s wr, 1.61k op/s 2024-04-23T18:37:19.651 INFO:tasks.workunit.client.0.smithi132.stdout:writing pattern 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-04-23T18:37:19.652 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-04-23T18:37:19.653 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-04-23T18:37:19.653 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-04-23T18:37:19.653 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-04-23T18:37:19.653 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-04-23T18:37:19.653 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-04-23T18:37:19.653 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-04-23T18:37:19.653 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-04-23T18:37:19.653 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-04-23T18:37:19.653 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-04-23T18:37:19.654 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-04-23T18:37:19.654 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-04-23T18:37:19.654 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-04-23T18:37:19.654 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-04-23T18:37:19.654 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-04-23T18:37:19.654 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-04-23T18:37:19.654 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-04-23T18:37:19.654 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-04-23T18:37:19.654 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-04-23T18:37:19.655 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-04-23T18:37:19.655 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-04-23T18:37:19.655 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-04-23T18:37:19.655 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-04-23T18:37:19.655 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-04-23T18:37:19.655 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-04-23T18:37:19.655 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-04-23T18:37:19.655 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-04-23T18:37:19.655 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-04-23T18:37:19.655 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-04-23T18:37:19.656 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-04-23T18:37:19.656 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-04-23T18:37:19.656 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-04-23T18:37:19.656 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-04-23T18:37:19.656 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-04-23T18:37:19.656 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-04-23T18:37:19.656 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-04-23T18:37:19.656 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-04-23T18:37:19.657 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-04-23T18:37:19.657 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-04-23T18:37:19.657 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-04-23T18:37:19.657 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-04-23T18:37:19.657 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-04-23T18:37:19.657 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-04-23T18:37:19.657 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-04-23T18:37:19.657 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-04-23T18:37:19.658 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-04-23T18:37:19.658 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-04-23T18:37:19.658 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-04-23T18:37:19.658 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-04-23T18:37:19.658 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-04-23T18:37:19.658 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-04-23T18:37:19.658 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-04-23T18:37:19.658 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-04-23T18:37:19.658 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-04-23T18:37:19.658 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-04-23T18:37:19.658 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-04-23T18:37:19.659 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-04-23T18:37:19.659 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-04-23T18:37:19.659 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-04-23T18:37:19.659 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-04-23T18:37:19.659 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-04-23T18:37:19.659 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-04-23T18:37:19.659 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-04-23T18:37:19.659 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-04-23T18:37:19.659 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-04-23T18:37:19.659 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-04-23T18:37:19.660 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-04-23T18:37:19.660 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-04-23T18:37:19.660 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-04-23T18:37:19.660 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-04-23T18:37:19.660 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-04-23T18:37:19.660 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-04-23T18:37:19.660 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-04-23T18:37:19.660 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-04-23T18:37:19.672 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-04-23T18:37:19.673 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-04-23T18:37:19.674 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-04-23T18:37:19.675 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-04-23T18:37:19.676 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-04-23T18:37:19.677 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-04-23T18:37:19.677 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-04-23T18:37:19.677 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-04-23T18:37:19.677 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-04-23T18:37:19.677 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-04-23T18:37:19.677 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-04-23T18:37:19.677 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-04-23T18:37:19.677 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-04-23T18:37:19.677 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-04-23T18:37:19.677 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-04-23T18:37:19.677 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-04-23T18:37:19.677 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-04-23T18:37:19.678 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-04-23T18:37:19.678 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-04-23T18:37:19.678 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-04-23T18:37:19.681 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-04-23T18:37:19.681 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-04-23T18:37:19.681 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-04-23T18:37:19.681 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-04-23T18:37:19.681 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-04-23T18:37:19.681 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-04-23T18:37:19.681 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-04-23T18:37:19.681 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-04-23T18:37:19.681 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-04-23T18:37:19.681 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-04-23T18:37:19.682 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-04-23T18:37:19.682 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-04-23T18:37:19.682 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-04-23T18:37:19.682 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-04-23T18:37:19.682 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-04-23T18:37:19.682 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-04-23T18:37:19.682 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-04-23T18:37:19.682 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-04-23T18:37:19.682 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-04-23T18:37:19.682 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-04-23T18:37:19.682 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-04-23T18:37:19.682 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-04-23T18:37:19.683 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-04-23T18:37:19.684 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-04-23T18:37:19.684 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-04-23T18:37:19.684 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-04-23T18:37:19.684 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-04-23T18:37:19.684 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-04-23T18:37:19.684 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-04-23T18:37:19.684 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-04-23T18:37:19.684 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-04-23T18:37:19.684 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-04-23T18:37:19.685 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-04-23T18:37:19.686 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-04-23T18:37:19.686 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-04-23T18:37:19.686 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-04-23T18:37:19.686 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-04-23T18:37:19.686 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-04-23T18:37:19.686 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-04-23T18:37:19.686 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-04-23T18:37:19.686 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-04-23T18:37:19.686 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-04-23T18:37:19.687 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-04-23T18:37:19.687 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-04-23T18:37:19.687 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-04-23T18:37:19.687 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-04-23T18:37:19.687 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-04-23T18:37:19.687 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-04-23T18:37:19.687 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-04-23T18:37:19.687 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-04-23T18:37:19.688 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-04-23T18:37:19.688 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-04-23T18:37:19.688 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-04-23T18:37:19.688 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-04-23T18:37:19.688 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-04-23T18:37:19.688 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-04-23T18:37:19.688 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-04-23T18:37:19.688 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-04-23T18:37:19.688 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-04-23T18:37:19.692 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-04-23T18:37:19.692 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-04-23T18:37:19.692 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-04-23T18:37:19.692 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-04-23T18:37:19.692 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-04-23T18:37:19.693 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-04-23T18:37:19.694 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-04-23T18:37:19.694 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-04-23T18:37:19.694 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-04-23T18:37:19.694 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-04-23T18:37:19.694 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-04-23T18:37:19.694 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-04-23T18:37:19.694 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-04-23T18:37:19.694 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-04-23T18:37:19.694 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-04-23T18:37:19.694 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-04-23T18:37:19.694 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-04-23T18:37:19.694 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-04-23T18:37:19.695 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-04-23T18:37:19.695 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-04-23T18:37:19.695 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-04-23T18:37:19.695 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-04-23T18:37:19.695 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-04-23T18:37:19.695 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-04-23T18:37:19.695 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-04-23T18:37:19.695 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-04-23T18:37:19.695 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-04-23T18:37:19.695 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-04-23T18:37:19.696 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-04-23T18:37:19.697 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-04-23T18:37:19.697 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-04-23T18:37:19.697 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-04-23T18:37:19.697 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-04-23T18:37:19.697 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-04-23T18:37:19.697 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-04-23T18:37:19.697 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-04-23T18:37:19.698 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-04-23T18:37:19.698 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-04-23T18:37:19.698 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-04-23T18:37:19.698 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-04-23T18:37:19.698 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-04-23T18:37:19.698 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-04-23T18:37:19.698 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-04-23T18:37:19.698 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-04-23T18:37:19.699 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-04-23T18:37:19.699 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-04-23T18:37:19.699 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-04-23T18:37:19.699 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-04-23T18:37:19.699 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-04-23T18:37:19.699 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-04-23T18:37:19.699 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-04-23T18:37:19.700 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-04-23T18:37:19.700 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-04-23T18:37:19.700 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-04-23T18:37:19.700 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-04-23T18:37:19.700 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-04-23T18:37:19.700 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-04-23T18:37:19.700 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-04-23T18:37:19.700 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-04-23T18:37:19.716 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-04-23T18:37:19.717 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-04-23T18:37:19.717 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-04-23T18:37:19.717 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-04-23T18:37:19.717 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-04-23T18:37:19.717 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-04-23T18:37:19.717 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-04-23T18:37:19.717 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-04-23T18:37:19.717 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-04-23T18:37:19.717 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-04-23T18:37:19.717 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-04-23T18:37:19.717 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-04-23T18:37:19.718 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-04-23T18:37:19.719 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-04-23T18:37:19.720 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-04-23T18:37:19.721 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-04-23T18:37:19.722 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-04-23T18:37:19.739 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-04-23T18:37:19.740 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-04-23T18:37:19.740 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-04-23T18:37:19.740 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-04-23T18:37:19.740 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-04-23T18:37:19.740 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-04-23T18:37:19.740 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-04-23T18:37:19.740 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-04-23T18:37:19.740 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-04-23T18:37:19.741 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-04-23T18:37:19.741 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-04-23T18:37:19.741 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-04-23T18:37:19.741 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-04-23T18:37:19.741 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-04-23T18:37:19.741 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-04-23T18:37:19.741 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-04-23T18:37:19.741 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-04-23T18:37:19.741 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-04-23T18:37:19.742 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-04-23T18:37:19.743 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-04-23T18:37:19.744 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-04-23T18:37:19.744 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-04-23T18:37:19.744 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-04-23T18:37:19.744 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-04-23T18:37:19.744 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-04-23T18:37:19.744 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-04-23T18:37:19.744 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-04-23T18:37:19.744 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-04-23T18:37:19.744 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-04-23T18:37:19.744 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-04-23T18:37:19.744 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-04-23T18:37:19.744 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-04-23T18:37:19.745 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-04-23T18:37:19.746 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-04-23T18:37:19.746 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-04-23T18:37:19.746 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-04-23T18:37:19.746 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-04-23T18:37:19.746 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-04-23T18:37:19.746 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-04-23T18:37:19.746 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-04-23T18:37:19.746 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-04-23T18:37:19.746 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-04-23T18:37:19.748 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-04-23T18:37:19.748 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-04-23T18:37:19.748 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-04-23T18:37:19.748 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-04-23T18:37:19.748 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-04-23T18:37:19.748 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-04-23T18:37:19.749 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-04-23T18:37:19.750 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-04-23T18:37:19.751 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-04-23T18:37:19.752 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-04-23T18:37:19.753 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-04-23T18:37:19.754 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-04-23T18:37:19.754 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-04-23T18:37:19.763 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-04-23T18:37:19.763 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-04-23T18:37:19.764 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-04-23T18:37:19.764 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-04-23T18:37:19.764 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-04-23T18:37:19.764 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-04-23T18:37:19.764 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-04-23T18:37:19.764 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-04-23T18:37:19.765 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-04-23T18:37:19.765 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-04-23T18:37:19.765 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-04-23T18:37:19.765 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-04-23T18:37:19.765 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-04-23T18:37:19.765 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-04-23T18:37:19.766 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-04-23T18:37:19.766 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-04-23T18:37:19.766 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-04-23T18:37:19.766 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-04-23T18:37:19.766 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-04-23T18:37:19.766 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-04-23T18:37:19.766 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-04-23T18:37:19.767 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-04-23T18:37:19.767 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-04-23T18:37:19.767 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-04-23T18:37:19.767 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-04-23T18:37:19.767 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-04-23T18:37:19.767 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-04-23T18:37:19.768 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-04-23T18:37:19.768 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-04-23T18:37:19.768 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-04-23T18:37:19.768 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-04-23T18:37:19.768 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-04-23T18:37:19.768 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-04-23T18:37:19.768 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-04-23T18:37:19.769 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-04-23T18:37:19.769 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-04-23T18:37:19.769 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-04-23T18:37:19.769 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-04-23T18:37:19.769 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-04-23T18:37:19.769 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-04-23T18:37:19.770 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-04-23T18:37:19.770 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-04-23T18:37:19.770 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-04-23T18:37:19.770 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-04-23T18:37:19.770 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-04-23T18:37:19.770 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-04-23T18:37:19.770 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-04-23T18:37:19.771 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-04-23T18:37:19.771 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-04-23T18:37:19.771 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-04-23T18:37:19.771 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-04-23T18:37:19.771 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-04-23T18:37:19.771 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-04-23T18:37:19.772 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-04-23T18:37:19.772 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-04-23T18:37:19.772 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-04-23T18:37:19.772 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-04-23T18:37:19.772 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-04-23T18:37:19.772 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-04-23T18:37:19.773 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-04-23T18:37:19.773 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-04-23T18:37:19.773 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-04-23T18:37:19.773 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-04-23T18:37:19.773 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-04-23T18:37:19.773 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-04-23T18:37:19.774 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-04-23T18:37:19.774 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-04-23T18:37:19.774 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-04-23T18:37:19.774 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-04-23T18:37:19.774 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-04-23T18:37:19.774 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-04-23T18:37:19.774 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-04-23T18:37:19.775 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-04-23T18:37:19.775 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-04-23T18:37:19.775 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-04-23T18:37:19.775 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-04-23T18:37:19.775 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-04-23T18:37:19.775 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-04-23T18:37:19.776 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-04-23T18:37:19.776 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-04-23T18:37:19.776 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-04-23T18:37:19.776 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-04-23T18:37:19.776 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-04-23T18:37:19.786 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-04-23T18:37:19.787 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-04-23T18:37:19.787 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-04-23T18:37:19.787 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-04-23T18:37:19.787 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-04-23T18:37:19.787 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-04-23T18:37:19.787 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-04-23T18:37:19.788 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-04-23T18:37:19.788 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-04-23T18:37:19.788 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-04-23T18:37:19.788 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-04-23T18:37:19.788 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-04-23T18:37:19.788 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-04-23T18:37:19.789 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-04-23T18:37:19.789 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-04-23T18:37:19.789 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-04-23T18:37:19.789 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-04-23T18:37:19.789 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-04-23T18:37:19.789 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-04-23T18:37:19.789 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-04-23T18:37:19.789 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-04-23T18:37:19.789 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-04-23T18:37:19.790 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-04-23T18:37:19.790 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-04-23T18:37:19.790 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-04-23T18:37:19.790 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-04-23T18:37:19.790 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-04-23T18:37:19.790 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-04-23T18:37:19.790 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-04-23T18:37:19.791 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-04-23T18:37:19.791 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-04-23T18:37:19.791 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-04-23T18:37:19.791 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-04-23T18:37:19.791 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-04-23T18:37:19.791 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-04-23T18:37:19.791 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-04-23T18:37:19.792 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-04-23T18:37:19.793 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-04-23T18:37:19.794 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-04-23T18:37:19.794 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-04-23T18:37:19.794 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-04-23T18:37:19.794 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-04-23T18:37:19.794 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-04-23T18:37:19.794 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-04-23T18:37:19.794 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-04-23T18:37:19.794 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-04-23T18:37:19.794 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-04-23T18:37:19.794 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-04-23T18:37:19.794 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-04-23T18:37:19.795 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-04-23T18:37:19.795 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-04-23T18:37:19.795 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-04-23T18:37:19.795 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-04-23T18:37:19.795 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-04-23T18:37:19.795 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-04-23T18:37:19.795 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-04-23T18:37:19.795 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-04-23T18:37:19.795 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-04-23T18:37:19.801 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-04-23T18:37:19.802 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-04-23T18:37:19.802 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-04-23T18:37:19.802 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-04-23T18:37:19.802 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-04-23T18:37:19.802 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-04-23T18:37:19.802 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-04-23T18:37:19.802 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-04-23T18:37:19.802 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-04-23T18:37:19.802 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-04-23T18:37:19.803 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-04-23T18:37:19.803 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-04-23T18:37:19.803 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-04-23T18:37:19.803 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-04-23T18:37:19.803 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-04-23T18:37:19.803 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-04-23T18:37:19.803 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-04-23T18:37:19.803 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-04-23T18:37:19.803 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-04-23T18:37:19.803 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-04-23T18:37:19.803 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-04-23T18:37:19.804 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-04-23T18:37:19.804 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-04-23T18:37:19.804 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-04-23T18:37:19.804 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-04-23T18:37:19.804 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-04-23T18:37:19.804 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-04-23T18:37:19.804 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-04-23T18:37:19.804 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-04-23T18:37:19.805 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-04-23T18:37:19.805 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-04-23T18:37:19.805 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-04-23T18:37:19.805 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-04-23T18:37:19.805 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-04-23T18:37:19.805 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-04-23T18:37:19.805 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-04-23T18:37:19.805 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-04-23T18:37:19.805 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-04-23T18:37:19.805 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-04-23T18:37:19.806 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-04-23T18:37:19.806 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-04-23T18:37:19.806 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-04-23T18:37:19.806 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-04-23T18:37:19.806 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-04-23T18:37:19.806 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-04-23T18:37:19.806 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-04-23T18:37:19.806 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-04-23T18:37:19.807 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-04-23T18:37:19.807 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-04-23T18:37:19.807 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-04-23T18:37:19.807 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-04-23T18:37:19.807 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-04-23T18:37:19.807 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-04-23T18:37:19.807 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-04-23T18:37:19.807 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-04-23T18:37:19.807 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-04-23T18:37:19.807 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-04-23T18:37:19.807 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-04-23T18:37:19.807 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-04-23T18:37:19.808 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-04-23T18:37:19.808 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-04-23T18:37:19.808 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-04-23T18:37:19.808 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-04-23T18:37:19.808 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-04-23T18:37:19.808 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-04-23T18:37:19.808 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-04-23T18:37:19.809 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-04-23T18:37:19.810 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-04-23T18:37:19.810 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-04-23T18:37:19.810 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-04-23T18:37:19.811 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-04-23T18:37:19.811 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-04-23T18:37:19.811 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-04-23T18:37:19.811 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-04-23T18:37:19.811 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-04-23T18:37:19.812 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-04-23T18:37:19.812 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-04-23T18:37:19.812 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-04-23T18:37:19.812 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-04-23T18:37:19.812 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-04-23T18:37:19.812 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-04-23T18:37:19.812 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-04-23T18:37:19.812 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-04-23T18:37:19.813 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-04-23T18:37:19.813 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-04-23T18:37:19.813 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-04-23T18:37:19.813 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-04-23T18:37:19.813 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-04-23T18:37:19.813 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-04-23T18:37:19.813 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-04-23T18:37:19.813 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-04-23T18:37:19.814 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-04-23T18:37:19.814 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-04-23T18:37:19.814 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-04-23T18:37:19.814 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-04-23T18:37:19.814 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-04-23T18:37:19.814 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-04-23T18:37:19.814 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-04-23T18:37:19.814 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-04-23T18:37:19.815 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-04-23T18:37:19.815 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-04-23T18:37:19.815 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-04-23T18:37:19.815 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-04-23T18:37:19.815 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-04-23T18:37:19.815 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-04-23T18:37:19.815 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-04-23T18:37:19.815 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-04-23T18:37:19.816 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-04-23T18:37:19.816 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-04-23T18:37:19.816 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-04-23T18:37:19.816 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-04-23T18:37:19.816 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-04-23T18:37:19.816 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-04-23T18:37:19.816 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-04-23T18:37:19.816 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-04-23T18:37:19.816 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-04-23T18:37:19.816 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-04-23T18:37:19.817 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-04-23T18:37:19.818 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-04-23T18:37:19.819 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-04-23T18:37:19.819 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-04-23T18:37:19.819 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-04-23T18:37:19.819 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-04-23T18:37:19.819 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-04-23T18:37:19.819 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-04-23T18:37:19.819 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-04-23T18:37:19.819 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-04-23T18:37:19.819 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-04-23T18:37:19.825 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-04-23T18:37:19.825 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-04-23T18:37:19.825 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-04-23T18:37:19.825 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-04-23T18:37:19.825 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-04-23T18:37:19.825 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-04-23T18:37:19.825 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-04-23T18:37:19.825 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-04-23T18:37:19.825 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-04-23T18:37:19.825 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-04-23T18:37:19.826 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-04-23T18:37:19.826 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-04-23T18:37:19.826 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-04-23T18:37:19.826 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-04-23T18:37:19.826 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-04-23T18:37:19.826 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-04-23T18:37:19.826 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-04-23T18:37:19.826 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-04-23T18:37:19.826 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-04-23T18:37:19.826 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-04-23T18:37:19.826 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-04-23T18:37:19.826 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-04-23T18:37:19.827 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-04-23T18:37:19.828 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-04-23T18:37:19.829 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-04-23T18:37:19.830 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-04-23T18:37:19.830 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-04-23T18:37:19.830 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-04-23T18:37:19.830 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-04-23T18:37:19.830 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-04-23T18:37:19.830 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-04-23T18:37:19.830 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-04-23T18:37:19.830 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-04-23T18:37:19.830 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-04-23T18:37:19.830 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-04-23T18:37:19.830 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-04-23T18:37:19.830 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-04-23T18:37:19.831 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-04-23T18:37:19.831 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-04-23T18:37:19.831 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-04-23T18:37:19.831 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-04-23T18:37:19.844 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-04-23T18:37:19.845 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-04-23T18:37:19.845 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-04-23T18:37:19.845 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-04-23T18:37:19.845 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-04-23T18:37:19.845 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-04-23T18:37:19.845 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-04-23T18:37:19.845 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-04-23T18:37:19.845 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-04-23T18:37:19.845 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-04-23T18:37:19.845 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-04-23T18:37:19.846 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-04-23T18:37:19.846 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-04-23T18:37:19.846 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-04-23T18:37:19.846 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-04-23T18:37:19.846 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-04-23T18:37:19.846 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-04-23T18:37:19.846 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-04-23T18:37:19.846 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-04-23T18:37:19.846 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-04-23T18:37:19.846 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-04-23T18:37:19.847 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-04-23T18:37:19.847 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-04-23T18:37:19.847 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-04-23T18:37:19.847 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-04-23T18:37:19.847 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-04-23T18:37:19.847 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-04-23T18:37:19.847 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-04-23T18:37:19.847 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-04-23T18:37:19.847 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-04-23T18:37:19.848 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-04-23T18:37:19.848 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-04-23T18:37:19.848 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-04-23T18:37:19.848 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-04-23T18:37:19.848 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-04-23T18:37:19.848 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-04-23T18:37:19.848 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-04-23T18:37:19.848 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-04-23T18:37:19.849 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-04-23T18:37:19.849 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-04-23T18:37:19.849 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-04-23T18:37:19.849 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-04-23T18:37:19.849 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-04-23T18:37:19.849 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-04-23T18:37:19.849 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-04-23T18:37:19.849 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-04-23T18:37:19.850 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-04-23T18:37:19.850 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-04-23T18:37:19.850 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-04-23T18:37:19.850 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-04-23T18:37:19.850 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-04-23T18:37:19.850 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-04-23T18:37:19.850 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-04-23T18:37:19.851 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-04-23T18:37:19.851 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-04-23T18:37:19.851 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-04-23T18:37:19.851 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-04-23T18:37:19.851 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-04-23T18:37:19.851 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-04-23T18:37:19.851 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-04-23T18:37:19.851 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-04-23T18:37:19.851 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-04-23T18:37:19.852 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-04-23T18:37:19.852 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-04-23T18:37:19.852 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-04-23T18:37:19.852 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-04-23T18:37:19.852 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-04-23T18:37:19.852 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-04-23T18:37:19.852 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-04-23T18:37:19.852 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-04-23T18:37:19.853 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-04-23T18:37:19.853 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-04-23T18:37:19.853 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-04-23T18:37:19.853 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-04-23T18:37:19.853 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-04-23T18:37:19.853 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-04-23T18:37:19.853 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-04-23T18:37:19.853 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-04-23T18:37:19.854 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-04-23T18:37:19.854 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-04-23T18:37:19.854 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-04-23T18:37:19.854 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-04-23T18:37:19.858 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-04-23T18:37:19.859 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-04-23T18:37:19.860 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-04-23T18:37:19.861 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-04-23T18:37:19.862 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-04-23T18:37:19.862 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-04-23T18:37:19.862 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-04-23T18:37:19.862 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-04-23T18:37:19.862 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-04-23T18:37:19.862 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-04-23T18:37:19.862 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-04-23T18:37:19.862 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-04-23T18:37:19.862 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-04-23T18:37:19.862 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-04-23T18:37:19.862 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-04-23T18:37:19.863 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-04-23T18:37:19.864 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-04-23T18:37:19.868 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-04-23T18:37:19.868 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-04-23T18:37:19.868 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-04-23T18:37:19.869 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-04-23T18:37:19.869 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-04-23T18:37:19.869 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-04-23T18:37:19.869 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-04-23T18:37:19.869 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-04-23T18:37:19.869 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-04-23T18:37:19.869 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-04-23T18:37:19.870 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-04-23T18:37:19.870 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-04-23T18:37:19.870 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-04-23T18:37:19.870 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-04-23T18:37:19.870 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-04-23T18:37:19.870 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-04-23T18:37:19.871 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-04-23T18:37:19.871 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-04-23T18:37:19.871 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-04-23T18:37:19.871 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-04-23T18:37:19.871 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-04-23T18:37:19.871 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-04-23T18:37:19.871 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-04-23T18:37:19.872 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-04-23T18:37:19.872 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-04-23T18:37:19.872 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-04-23T18:37:19.872 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-04-23T18:37:19.872 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-04-23T18:37:19.872 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-04-23T18:37:19.872 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-04-23T18:37:19.873 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-04-23T18:37:19.873 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-04-23T18:37:19.873 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-04-23T18:37:19.873 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-04-23T18:37:19.873 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-04-23T18:37:19.873 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-04-23T18:37:19.874 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-04-23T18:37:19.874 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-04-23T18:37:19.874 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-04-23T18:37:19.874 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-04-23T18:37:19.874 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-04-23T18:37:19.874 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-04-23T18:37:19.874 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-04-23T18:37:19.875 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-04-23T18:37:19.875 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-04-23T18:37:19.875 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-04-23T18:37:19.875 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-04-23T18:37:19.875 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-04-23T18:37:19.875 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-04-23T18:37:19.876 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-04-23T18:37:19.876 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-04-23T18:37:19.876 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-04-23T18:37:19.876 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-04-23T18:37:19.876 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-04-23T18:37:19.876 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-04-23T18:37:19.877 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-04-23T18:37:19.877 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-04-23T18:37:19.877 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-04-23T18:37:19.877 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-04-23T18:37:19.877 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-04-23T18:37:19.877 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-04-23T18:37:19.877 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-04-23T18:37:19.878 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-04-23T18:37:19.878 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-04-23T18:37:19.878 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-04-23T18:37:19.878 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-04-23T18:37:19.878 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-04-23T18:37:19.878 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-04-23T18:37:19.879 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-04-23T18:37:19.879 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-04-23T18:37:19.879 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-04-23T18:37:19.879 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-04-23T18:37:19.879 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-04-23T18:37:19.879 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-04-23T18:37:19.880 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-04-23T18:37:19.880 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-04-23T18:37:19.880 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-04-23T18:37:19.880 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-04-23T18:37:19.880 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-04-23T18:37:19.880 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-04-23T18:37:19.880 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-04-23T18:37:19.881 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-04-23T18:37:19.884 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-04-23T18:37:19.885 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-04-23T18:37:19.885 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-04-23T18:37:19.885 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-04-23T18:37:19.885 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-04-23T18:37:19.885 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-04-23T18:37:19.885 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-04-23T18:37:19.886 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-04-23T18:37:19.886 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-04-23T18:37:19.886 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-04-23T18:37:19.886 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-04-23T18:37:19.886 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-04-23T18:37:19.886 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-04-23T18:37:19.887 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-04-23T18:37:19.887 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-04-23T18:37:19.887 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-04-23T18:37:19.887 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-04-23T18:37:19.887 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-04-23T18:37:19.887 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-04-23T18:37:19.887 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-04-23T18:37:19.888 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-04-23T18:37:19.888 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-04-23T18:37:19.888 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-04-23T18:37:19.888 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-04-23T18:37:19.888 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-04-23T18:37:19.888 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-04-23T18:37:19.889 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-04-23T18:37:19.889 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-04-23T18:37:19.889 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-04-23T18:37:19.889 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-04-23T18:37:19.889 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-04-23T18:37:19.889 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-04-23T18:37:19.889 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-04-23T18:37:19.890 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-04-23T18:37:19.890 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-04-23T18:37:19.890 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-04-23T18:37:19.890 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-04-23T18:37:19.890 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-04-23T18:37:19.890 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-04-23T18:37:19.891 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-04-23T18:37:19.891 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-04-23T18:37:19.891 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-04-23T18:37:19.891 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-04-23T18:37:19.891 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-04-23T18:37:19.891 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-04-23T18:37:19.891 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-04-23T18:37:19.892 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-04-23T18:37:19.892 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-04-23T18:37:19.892 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-04-23T18:37:19.892 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-04-23T18:37:19.892 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-04-23T18:37:19.892 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-04-23T18:37:19.892 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-04-23T18:37:19.892 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-04-23T18:37:19.893 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-04-23T18:37:19.893 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-04-23T18:37:19.893 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-04-23T18:37:19.893 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-04-23T18:37:19.893 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-04-23T18:37:19.893 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-04-23T18:37:19.893 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-04-23T18:37:19.893 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-04-23T18:37:19.894 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-04-23T18:37:19.894 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-04-23T18:37:19.894 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-04-23T18:37:19.894 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-04-23T18:37:19.894 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-04-23T18:37:19.894 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-04-23T18:37:19.894 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-04-23T18:37:19.894 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-04-23T18:37:19.894 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-04-23T18:37:19.895 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-04-23T18:37:19.895 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-04-23T18:37:19.895 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-04-23T18:37:19.895 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-04-23T18:37:19.895 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-04-23T18:37:19.895 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-04-23T18:37:19.895 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-04-23T18:37:19.895 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-04-23T18:37:19.896 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-04-23T18:37:19.896 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-04-23T18:37:19.896 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-04-23T18:37:19.896 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-04-23T18:37:19.910 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-04-23T18:37:19.910 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-04-23T18:37:19.910 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-04-23T18:37:19.910 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-04-23T18:37:19.910 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-04-23T18:37:19.910 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-04-23T18:37:19.910 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-04-23T18:37:19.910 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-04-23T18:37:19.911 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-04-23T18:37:19.912 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-04-23T18:37:19.912 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-04-23T18:37:19.912 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-04-23T18:37:19.912 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-04-23T18:37:19.912 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-04-23T18:37:19.912 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-04-23T18:37:19.912 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-04-23T18:37:19.912 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-04-23T18:37:19.912 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-04-23T18:37:19.913 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-04-23T18:37:19.914 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-04-23T18:37:19.914 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-04-23T18:37:19.914 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-04-23T18:37:19.914 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-04-23T18:37:19.914 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-04-23T18:37:19.914 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-04-23T18:37:19.914 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-04-23T18:37:19.914 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-04-23T18:37:19.914 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-04-23T18:37:19.915 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-04-23T18:37:19.916 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-04-23T18:37:19.916 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-04-23T18:37:19.916 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-04-23T18:37:19.916 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-04-23T18:37:19.916 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-04-23T18:37:19.916 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-04-23T18:37:19.916 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-04-23T18:37:19.916 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-04-23T18:37:19.916 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-04-23T18:37:19.917 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-04-23T18:37:19.917 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-04-23T18:37:19.923 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-04-23T18:37:19.923 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-04-23T18:37:19.923 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-04-23T18:37:19.923 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-04-23T18:37:19.923 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-04-23T18:37:19.923 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-04-23T18:37:19.923 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-04-23T18:37:19.924 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-04-23T18:37:19.924 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-04-23T18:37:19.924 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-04-23T18:37:19.924 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-04-23T18:37:19.924 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-04-23T18:37:19.924 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-04-23T18:37:19.924 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-04-23T18:37:19.924 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-04-23T18:37:19.925 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-04-23T18:37:19.925 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-04-23T18:37:19.925 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-04-23T18:37:19.925 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-04-23T18:37:19.925 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-04-23T18:37:19.925 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-04-23T18:37:19.925 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-04-23T18:37:19.926 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-04-23T18:37:19.926 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-04-23T18:37:19.926 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-04-23T18:37:19.926 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-04-23T18:37:19.926 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-04-23T18:37:19.926 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-04-23T18:37:19.926 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-04-23T18:37:19.927 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-04-23T18:37:19.928 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-04-23T18:37:19.928 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-04-23T18:37:19.928 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-04-23T18:37:19.928 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-04-23T18:37:19.928 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-04-23T18:37:19.928 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-04-23T18:37:19.928 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-04-23T18:37:19.928 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-04-23T18:37:19.929 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-04-23T18:37:19.929 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-04-23T18:37:19.929 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-04-23T18:37:19.929 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-04-23T18:37:19.929 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-04-23T18:37:19.929 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-04-23T18:37:19.929 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-04-23T18:37:19.929 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-04-23T18:37:19.929 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-04-23T18:37:19.930 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-04-23T18:37:19.930 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-04-23T18:37:19.930 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-04-23T18:37:19.930 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-04-23T18:37:19.930 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-04-23T18:37:19.930 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-04-23T18:37:19.930 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-04-23T18:37:19.930 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-04-23T18:37:19.931 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-04-23T18:37:19.931 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-04-23T18:37:19.931 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-04-23T18:37:19.931 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-04-23T18:37:19.931 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-04-23T18:37:19.931 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-04-23T18:37:19.931 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-04-23T18:37:19.931 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-04-23T18:37:19.931 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-04-23T18:37:19.931 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-04-23T18:37:19.932 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-04-23T18:37:19.932 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-04-23T18:37:19.932 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-04-23T18:37:19.932 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-04-23T18:37:19.934 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-04-23T18:37:19.934 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-04-23T18:37:19.934 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-04-23T18:37:19.934 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-04-23T18:37:19.934 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-04-23T18:37:19.934 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-04-23T18:37:19.935 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-04-23T18:37:19.936 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-04-23T18:37:19.937 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-04-23T18:37:19.938 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-04-23T18:37:19.939 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-04-23T18:37:19.940 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-04-23T18:37:19.940 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-04-23T18:37:19.940 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-04-23T18:37:19.940 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-04-23T18:37:19.951 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-04-23T18:37:19.951 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-04-23T18:37:19.951 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-04-23T18:37:19.951 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-04-23T18:37:19.951 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-04-23T18:37:19.951 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-04-23T18:37:19.952 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-04-23T18:37:19.952 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-04-23T18:37:19.952 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-04-23T18:37:19.952 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-04-23T18:37:19.952 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-04-23T18:37:19.952 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-04-23T18:37:19.952 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-04-23T18:37:19.952 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-04-23T18:37:19.952 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-04-23T18:37:19.952 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-04-23T18:37:19.952 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-04-23T18:37:19.952 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-04-23T18:37:19.953 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-04-23T18:37:19.953 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-04-23T18:37:19.953 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-04-23T18:37:19.953 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-04-23T18:37:19.953 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-04-23T18:37:19.953 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-04-23T18:37:19.953 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-04-23T18:37:19.954 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-04-23T18:37:19.954 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-04-23T18:37:19.954 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-04-23T18:37:19.954 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-04-23T18:37:19.954 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-04-23T18:37:19.954 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-04-23T18:37:19.954 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-04-23T18:37:19.954 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-04-23T18:37:19.954 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-04-23T18:37:19.955 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-04-23T18:37:19.956 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-04-23T18:37:19.956 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-04-23T18:37:19.956 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-04-23T18:37:19.956 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-04-23T18:37:19.956 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-04-23T18:37:19.956 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-04-23T18:37:19.956 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-04-23T18:37:19.956 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-04-23T18:37:19.956 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-04-23T18:37:19.956 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-04-23T18:37:19.956 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-04-23T18:37:19.957 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-04-23T18:37:19.957 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-04-23T18:37:19.957 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-04-23T18:37:19.957 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-04-23T18:37:19.957 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-04-23T18:37:19.957 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-04-23T18:37:19.957 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-04-23T18:37:19.957 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-04-23T18:37:19.957 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-04-23T18:37:19.957 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-04-23T18:37:19.958 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-04-23T18:37:19.958 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-04-23T18:37:19.958 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-04-23T18:37:19.958 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-04-23T18:37:19.958 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-04-23T18:37:19.958 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-04-23T18:37:19.958 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-04-23T18:37:19.958 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-04-23T18:37:19.958 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-04-23T18:37:19.959 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-04-23T18:37:19.959 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-04-23T18:37:19.959 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-04-23T18:37:19.959 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-04-23T18:37:19.959 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-04-23T18:37:19.975 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-04-23T18:37:19.975 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-04-23T18:37:19.976 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-04-23T18:37:19.976 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-04-23T18:37:19.976 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-04-23T18:37:19.976 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-04-23T18:37:19.976 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-04-23T18:37:19.976 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-04-23T18:37:19.976 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-04-23T18:37:19.976 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-04-23T18:37:19.976 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-04-23T18:37:19.976 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-04-23T18:37:19.977 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-04-23T18:37:19.977 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-04-23T18:37:19.977 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-04-23T18:37:19.977 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-04-23T18:37:19.977 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-04-23T18:37:19.977 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-04-23T18:37:19.977 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-04-23T18:37:19.977 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-04-23T18:37:19.977 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-04-23T18:37:19.977 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-04-23T18:37:19.977 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-04-23T18:37:19.977 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-04-23T18:37:19.978 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-04-23T18:37:19.979 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-04-23T18:37:19.979 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-04-23T18:37:19.979 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-04-23T18:37:19.979 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-04-23T18:37:19.979 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-04-23T18:37:19.979 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-04-23T18:37:19.979 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-04-23T18:37:19.980 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-04-23T18:37:19.981 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-04-23T18:37:19.981 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-04-23T18:37:19.981 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-04-23T18:37:19.981 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-04-23T18:37:19.981 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-04-23T18:37:19.981 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-04-23T18:37:19.981 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-04-23T18:37:19.981 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-04-23T18:37:19.981 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-04-23T18:37:19.982 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-04-23T18:37:19.982 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-04-23T18:37:19.982 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-04-23T18:37:19.982 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-04-23T18:37:19.982 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-04-23T18:37:19.982 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-04-23T18:37:19.982 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-04-23T18:37:19.982 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-04-23T18:37:19.982 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-04-23T18:37:19.982 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-04-23T18:37:19.982 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-04-23T18:37:19.982 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-04-23T18:37:19.983 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-04-23T18:37:19.983 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-04-23T18:37:19.983 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-04-23T18:37:19.983 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-04-23T18:37:19.984 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-04-23T18:37:19.985 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-04-23T18:37:19.985 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-04-23T18:37:19.985 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-04-23T18:37:19.985 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-04-23T18:37:19.985 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-04-23T18:37:19.985 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-04-23T18:37:19.985 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-04-23T18:37:19.985 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-04-23T18:37:19.985 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-04-23T18:37:19.986 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-04-23T18:37:19.986 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-04-23T18:37:19.986 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-04-23T18:37:19.986 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-04-23T18:37:19.986 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-04-23T18:37:19.986 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-04-23T18:37:19.986 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-04-23T18:37:19.986 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-04-23T18:37:19.986 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-04-23T18:37:19.987 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-04-23T18:37:19.987 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-04-23T18:37:19.987 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-04-23T18:37:19.987 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-04-23T18:37:19.987 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-04-23T18:37:19.987 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-04-23T18:37:19.987 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-04-23T18:37:19.987 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-04-23T18:37:19.987 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-04-23T18:37:19.988 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-04-23T18:37:19.988 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-04-23T18:37:19.988 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-04-23T18:37:19.988 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-04-23T18:37:19.988 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-04-23T18:37:19.988 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-04-23T18:37:19.988 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-04-23T18:37:19.988 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-04-23T18:37:19.988 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-04-23T18:37:19.988 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-04-23T18:37:19.988 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-04-23T18:37:19.989 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-04-23T18:37:19.989 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-04-23T18:37:19.989 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-04-23T18:37:19.989 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-04-23T18:37:19.989 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-04-23T18:37:19.989 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-04-23T18:37:19.989 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-04-23T18:37:19.989 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-04-23T18:37:19.989 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-04-23T18:37:19.989 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-04-23T18:37:19.989 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-04-23T18:37:19.990 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-04-23T18:37:19.990 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-04-23T18:37:19.990 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-04-23T18:37:19.990 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-04-23T18:37:19.990 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-04-23T18:37:19.990 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-04-23T18:37:19.990 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-04-23T18:37:19.990 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-04-23T18:37:19.990 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-04-23T18:37:19.990 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-04-23T18:37:19.990 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-04-23T18:37:19.990 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-04-23T18:37:19.991 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-04-23T18:37:19.991 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-04-23T18:37:19.991 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-04-23T18:37:19.991 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-04-23T18:37:19.991 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-04-23T18:37:19.991 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-04-23T18:37:19.991 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-04-23T18:37:19.991 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-04-23T18:37:19.991 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-04-23T18:37:19.991 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-04-23T18:37:19.992 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-04-23T18:37:19.992 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-04-23T18:37:19.992 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-04-23T18:37:19.992 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-04-23T18:37:19.992 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-04-23T18:37:19.992 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-04-23T18:37:19.992 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-04-23T18:37:19.992 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-04-23T18:37:19.992 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-04-23T18:37:19.992 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-04-23T18:37:19.994 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-04-23T18:37:19.994 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-04-23T18:37:19.995 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-04-23T18:37:19.996 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-04-23T18:37:19.997 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-04-23T18:37:19.997 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-04-23T18:37:19.997 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-04-23T18:37:19.997 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-04-23T18:37:19.997 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-04-23T18:37:19.997 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-04-23T18:37:19.997 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-04-23T18:37:19.997 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-04-23T18:37:19.997 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-04-23T18:37:19.997 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-04-23T18:37:19.998 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-04-23T18:37:19.998 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-04-23T18:37:19.998 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-04-23T18:37:19.998 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-04-23T18:37:19.998 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-04-23T18:37:19.998 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-04-23T18:37:19.998 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-04-23T18:37:19.998 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-04-23T18:37:19.998 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-04-23T18:37:19.999 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-04-23T18:37:19.999 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-04-23T18:37:19.999 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-04-23T18:37:19.999 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-04-23T18:37:19.999 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-04-23T18:37:19.999 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-04-23T18:37:19.999 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-04-23T18:37:19.999 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-04-23T18:37:19.999 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-04-23T18:37:19.999 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-04-23T18:37:19.999 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-04-23T18:37:19.999 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-04-23T18:37:20.000 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-04-23T18:37:20.000 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-04-23T18:37:20.000 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-04-23T18:37:20.000 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-04-23T18:37:20.000 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-04-23T18:37:20.000 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-04-23T18:37:20.000 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-04-23T18:37:20.000 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-04-23T18:37:20.000 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-04-23T18:37:20.000 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-04-23T18:37:20.000 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-04-23T18:37:20.000 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-04-23T18:37:20.001 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-04-23T18:37:20.001 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-04-23T18:37:20.001 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-04-23T18:37:20.001 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-04-23T18:37:20.001 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-04-23T18:37:20.001 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-04-23T18:37:20.001 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-04-23T18:37:20.015 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-04-23T18:37:20.015 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-04-23T18:37:20.015 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-04-23T18:37:20.015 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-04-23T18:37:20.015 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-04-23T18:37:20.015 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-04-23T18:37:20.015 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-04-23T18:37:20.015 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-04-23T18:37:20.015 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-04-23T18:37:20.015 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-04-23T18:37:20.015 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-04-23T18:37:20.016 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-04-23T18:37:20.017 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-04-23T18:37:20.018 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-04-23T18:37:20.019 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-04-23T18:37:20.020 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-04-23T18:37:20.035 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-04-23T18:37:20.035 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-04-23T18:37:20.035 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-04-23T18:37:20.035 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-04-23T18:37:20.035 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-04-23T18:37:20.035 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-04-23T18:37:20.036 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-04-23T18:37:20.037 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-04-23T18:37:20.037 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-04-23T18:37:20.037 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-04-23T18:37:20.037 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-04-23T18:37:20.037 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-04-23T18:37:20.037 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-04-23T18:37:20.037 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-04-23T18:37:20.037 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-04-23T18:37:20.037 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-04-23T18:37:20.037 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-04-23T18:37:20.037 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-04-23T18:37:20.038 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-04-23T18:37:20.038 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-04-23T18:37:20.038 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-04-23T18:37:20.038 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-04-23T18:37:20.038 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-04-23T18:37:20.038 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-04-23T18:37:20.038 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-04-23T18:37:20.038 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-04-23T18:37:20.038 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-04-23T18:37:20.038 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-04-23T18:37:20.039 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-04-23T18:37:20.040 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-04-23T18:37:20.040 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-04-23T18:37:20.040 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-04-23T18:37:20.040 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-04-23T18:37:20.040 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-04-23T18:37:20.040 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-04-23T18:37:20.040 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-04-23T18:37:20.040 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-04-23T18:37:20.041 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-04-23T18:37:20.041 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-04-23T18:37:20.041 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-04-23T18:37:20.041 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-04-23T18:37:20.041 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-04-23T18:37:20.041 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-04-23T18:37:20.041 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-04-23T18:37:20.041 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-04-23T18:37:20.041 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-04-23T18:37:20.041 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-04-23T18:37:20.042 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-04-23T18:37:20.042 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-04-23T18:37:20.042 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-04-23T18:37:20.042 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-04-23T18:37:20.042 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-04-23T18:37:20.042 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-04-23T18:37:20.042 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-04-23T18:37:20.042 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-04-23T18:37:20.043 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-04-23T18:37:20.044 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-04-23T18:37:20.045 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-04-23T18:37:20.046 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-04-23T18:37:20.047 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-04-23T18:37:20.047 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-04-23T18:37:20.047 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-04-23T18:37:20.047 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-04-23T18:37:20.047 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-04-23T18:37:20.047 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-04-23T18:37:20.047 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-04-23T18:37:20.047 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-04-23T18:37:20.047 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-04-23T18:37:20.048 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-04-23T18:37:20.049 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-04-23T18:37:20.049 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-04-23T18:37:20.049 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-04-23T18:37:20.049 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-04-23T18:37:20.049 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-04-23T18:37:20.049 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-04-23T18:37:20.049 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-04-23T18:37:20.049 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-04-23T18:37:20.049 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-04-23T18:37:20.050 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-04-23T18:37:20.050 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-04-23T18:37:20.050 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-04-23T18:37:20.050 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-04-23T18:37:20.050 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-04-23T18:37:20.050 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-04-23T18:37:20.050 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-04-23T18:37:20.050 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-04-23T18:37:20.051 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-04-23T18:37:20.051 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-04-23T18:37:20.051 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-04-23T18:37:20.051 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-04-23T18:37:20.051 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-04-23T18:37:20.051 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-04-23T18:37:20.051 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-04-23T18:37:20.051 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-04-23T18:37:20.051 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-04-23T18:37:20.052 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-04-23T18:37:20.052 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-04-23T18:37:20.052 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-04-23T18:37:20.052 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-04-23T18:37:20.052 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-04-23T18:37:20.052 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-04-23T18:37:20.052 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-04-23T18:37:20.053 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-04-23T18:37:20.053 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-04-23T18:37:20.054 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-04-23T18:37:20.054 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-04-23T18:37:20.054 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-04-23T18:37:20.054 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-04-23T18:37:20.054 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-04-23T18:37:20.054 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-04-23T18:37:20.054 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-04-23T18:37:20.054 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-04-23T18:37:20.054 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-04-23T18:37:20.054 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-04-23T18:37:20.054 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-04-23T18:37:20.054 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-04-23T18:37:20.055 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-04-23T18:37:20.055 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-04-23T18:37:20.055 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-04-23T18:37:20.055 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-04-23T18:37:20.055 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-04-23T18:37:20.055 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-04-23T18:37:20.055 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-04-23T18:37:20.055 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-04-23T18:37:20.055 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-04-23T18:37:20.055 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-04-23T18:37:20.056 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-04-23T18:37:20.057 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-04-23T18:37:20.057 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-04-23T18:37:20.057 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-04-23T18:37:20.057 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-04-23T18:37:20.057 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-04-23T18:37:20.057 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-04-23T18:37:20.057 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-04-23T18:37:20.057 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-04-23T18:37:20.057 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-04-23T18:37:20.057 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-04-23T18:37:20.058 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-04-23T18:37:20.058 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-04-23T18:37:20.058 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-04-23T18:37:20.058 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-04-23T18:37:20.058 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-04-23T18:37:20.058 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-04-23T18:37:20.058 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-04-23T18:37:20.058 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-04-23T18:37:20.058 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-04-23T18:37:20.058 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-04-23T18:37:20.058 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-04-23T18:37:20.059 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-04-23T18:37:20.060 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-04-23T18:37:20.060 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-04-23T18:37:20.060 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-04-23T18:37:20.060 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-04-23T18:37:20.060 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-04-23T18:37:20.060 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-04-23T18:37:20.060 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-04-23T18:37:20.060 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-04-23T18:37:20.060 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-04-23T18:37:20.060 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-04-23T18:37:20.061 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-04-23T18:37:20.071 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-04-23T18:37:20.071 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-04-23T18:37:20.071 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-04-23T18:37:20.071 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-04-23T18:37:20.071 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-04-23T18:37:20.071 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-04-23T18:37:20.071 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-04-23T18:37:20.072 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-04-23T18:37:20.073 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-04-23T18:37:20.074 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-04-23T18:37:20.074 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-04-23T18:37:20.074 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-04-23T18:37:20.074 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-04-23T18:37:20.074 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-04-23T18:37:20.074 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-04-23T18:37:20.074 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-04-23T18:37:20.074 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-04-23T18:37:20.074 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-04-23T18:37:20.074 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-04-23T18:37:20.075 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-04-23T18:37:20.076 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-04-23T18:37:20.076 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-04-23T18:37:20.076 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-04-23T18:37:20.076 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-04-23T18:37:20.076 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-04-23T18:37:20.076 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-04-23T18:37:20.076 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-04-23T18:37:20.076 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-04-23T18:37:20.076 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-04-23T18:37:20.076 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-04-23T18:37:20.076 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-04-23T18:37:20.077 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-04-23T18:37:20.077 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-04-23T18:37:20.077 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-04-23T18:37:20.077 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-04-23T18:37:20.077 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-04-23T18:37:20.077 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-04-23T18:37:20.077 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-04-23T18:37:20.077 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-04-23T18:37:20.077 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-04-23T18:37:20.077 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-04-23T18:37:20.077 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-04-23T18:37:20.077 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-04-23T18:37:20.090 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-04-23T18:37:20.090 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-04-23T18:37:20.090 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-04-23T18:37:20.090 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-04-23T18:37:20.090 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-04-23T18:37:20.090 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-04-23T18:37:20.090 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-04-23T18:37:20.090 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-04-23T18:37:20.090 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-04-23T18:37:20.091 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-04-23T18:37:20.091 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-04-23T18:37:20.091 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-04-23T18:37:20.091 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-04-23T18:37:20.091 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-04-23T18:37:20.091 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-04-23T18:37:20.091 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-04-23T18:37:20.091 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-04-23T18:37:20.091 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-04-23T18:37:20.091 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-04-23T18:37:20.091 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-04-23T18:37:20.092 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-04-23T18:37:20.093 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-04-23T18:37:20.094 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-04-23T18:37:20.094 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-04-23T18:37:20.094 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-04-23T18:37:20.094 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-04-23T18:37:20.094 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-04-23T18:37:20.094 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-04-23T18:37:20.094 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-04-23T18:37:20.094 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-04-23T18:37:20.095 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-04-23T18:37:20.095 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-04-23T18:37:20.095 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-04-23T18:37:20.095 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-04-23T18:37:20.095 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-04-23T18:37:20.095 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-04-23T18:37:20.095 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-04-23T18:37:20.095 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-04-23T18:37:20.095 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-04-23T18:37:20.096 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-04-23T18:37:20.096 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-04-23T18:37:20.096 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-04-23T18:37:20.096 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-04-23T18:37:20.096 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-04-23T18:37:20.096 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-04-23T18:37:20.096 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-04-23T18:37:20.096 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-04-23T18:37:20.096 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-04-23T18:37:20.096 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-04-23T18:37:20.096 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-04-23T18:37:20.096 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-04-23T18:37:20.097 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-04-23T18:37:20.097 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-04-23T18:37:20.097 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-04-23T18:37:20.098 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-04-23T18:37:20.098 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-04-23T18:37:20.098 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-04-23T18:37:20.098 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-04-23T18:37:20.099 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-04-23T18:37:20.099 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-04-23T18:37:20.099 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-04-23T18:37:20.099 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-04-23T18:37:20.099 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-04-23T18:37:20.099 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-04-23T18:37:20.099 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-04-23T18:37:20.099 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-04-23T18:37:20.099 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-04-23T18:37:20.099 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-04-23T18:37:20.099 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-04-23T18:37:20.100 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-04-23T18:37:20.100 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-04-23T18:37:20.100 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-04-23T18:37:20.100 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-04-23T18:37:20.100 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-04-23T18:37:20.100 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-04-23T18:37:20.100 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-04-23T18:37:20.100 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-04-23T18:37:20.100 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-04-23T18:37:20.100 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-04-23T18:37:20.101 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-04-23T18:37:20.101 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-04-23T18:37:20.101 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-04-23T18:37:20.101 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-04-23T18:37:20.101 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-04-23T18:37:20.101 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-04-23T18:37:20.101 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-04-23T18:37:20.101 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-04-23T18:37:20.101 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-04-23T18:37:20.101 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-04-23T18:37:20.101 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-04-23T18:37:20.102 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-04-23T18:37:20.102 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-04-23T18:37:20.102 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-04-23T18:37:20.102 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-04-23T18:37:20.102 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-04-23T18:37:20.102 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-04-23T18:37:20.102 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-04-23T18:37:20.102 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-04-23T18:37:20.102 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-04-23T18:37:20.103 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-04-23T18:37:20.103 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-04-23T18:37:20.103 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-04-23T18:37:20.103 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-04-23T18:37:20.103 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-04-23T18:37:20.103 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-04-23T18:37:20.103 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-04-23T18:37:20.103 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-04-23T18:37:20.103 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-04-23T18:37:20.104 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-04-23T18:37:20.104 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-04-23T18:37:20.104 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-04-23T18:37:20.104 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-04-23T18:37:20.104 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-04-23T18:37:20.104 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-04-23T18:37:20.104 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-04-23T18:37:20.104 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-04-23T18:37:20.105 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-04-23T18:37:20.105 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-04-23T18:37:20.105 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-04-23T18:37:20.105 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-04-23T18:37:20.105 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-04-23T18:37:20.105 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-04-23T18:37:20.105 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-04-23T18:37:20.105 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-04-23T18:37:20.106 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-04-23T18:37:20.106 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-04-23T18:37:20.106 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-04-23T18:37:20.106 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-04-23T18:37:20.106 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-04-23T18:37:20.106 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-04-23T18:37:20.107 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-04-23T18:37:20.107 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-04-23T18:37:20.107 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-04-23T18:37:20.107 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-04-23T18:37:20.107 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-04-23T18:37:20.107 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-04-23T18:37:20.282 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-04-23T18:37:20.282 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-04-23T18:37:20.282 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-04-23T18:37:20.282 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-04-23T18:37:20.283 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-04-23T18:37:20.283 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-04-23T18:37:20.283 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-04-23T18:37:20.283 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-04-23T18:37:20.283 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-04-23T18:37:20.283 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-04-23T18:37:20.283 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-04-23T18:37:20.284 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-04-23T18:37:20.284 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-04-23T18:37:20.284 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-04-23T18:37:20.284 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-04-23T18:37:20.284 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-04-23T18:37:20.284 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-04-23T18:37:20.284 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-04-23T18:37:20.284 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-04-23T18:37:20.284 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-04-23T18:37:20.285 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-04-23T18:37:20.285 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-04-23T18:37:20.285 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-04-23T18:37:20.285 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-04-23T18:37:20.285 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-04-23T18:37:20.285 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-04-23T18:37:20.285 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-04-23T18:37:20.285 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-04-23T18:37:20.285 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-04-23T18:37:20.285 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-04-23T18:37:20.285 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-04-23T18:37:20.286 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-04-23T18:37:20.286 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-04-23T18:37:20.286 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-04-23T18:37:20.286 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-04-23T18:37:20.286 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-04-23T18:37:20.286 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-04-23T18:37:20.286 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-04-23T18:37:20.286 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-04-23T18:37:20.287 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-04-23T18:37:20.287 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-04-23T18:37:20.287 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-04-23T18:37:20.287 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-04-23T18:37:20.287 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-04-23T18:37:20.287 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-04-23T18:37:20.287 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-04-23T18:37:20.287 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-04-23T18:37:20.287 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-04-23T18:37:20.288 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-04-23T18:37:20.288 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-04-23T18:37:20.288 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-04-23T18:37:20.288 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-04-23T18:37:20.288 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-04-23T18:37:20.288 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-04-23T18:37:20.288 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-04-23T18:37:20.288 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-04-23T18:37:20.288 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-04-23T18:37:20.289 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-04-23T18:37:20.289 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-04-23T18:37:20.289 INFO:tasks.workunit.client.0.smithi132.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-04-23T18:37:20.289 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-04-23T18:37:20.289 INFO:tasks.workunit.client.0.smithi132.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-04-23T18:37:20.289 INFO:tasks.workunit.client.0.smithi132.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-04-23T18:37:20.289 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-04-23T18:37:20.289 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-04-23T18:37:20.290 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-04-23T18:37:20.290 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-04-23T18:37:20.290 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-04-23T18:37:20.290 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-04-23T18:37:20.290 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-04-23T18:37:20.290 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-04-23T18:37:20.290 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-04-23T18:37:20.290 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-04-23T18:37:20.291 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-04-23T18:37:20.291 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-04-23T18:37:20.291 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-04-23T18:37:20.291 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-04-23T18:37:20.291 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-04-23T18:37:20.291 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-04-23T18:37:20.291 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-04-23T18:37:20.291 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-04-23T18:37:20.291 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-04-23T18:37:20.291 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-04-23T18:37:20.291 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-04-23T18:37:20.411 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:20 smithi148 ceph-mon[32311]: pgmap v41: 97 pgs: 97 active+clean; 2.0 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 2.71k op/s 2024-04-23T18:37:20.412 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:20 smithi132 ceph-mon[29178]: pgmap v41: 97 pgs: 97 active+clean; 2.0 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 2.71k op/s 2024-04-23T18:37:20.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:20 smithi157 ceph-mon[32580]: pgmap v41: 97 pgs: 97 active+clean; 2.0 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 2.71k op/s 2024-04-23T18:37:20.919 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-04-23T18:37:20.919 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-04-23T18:37:20.919 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-04-23T18:37:20.919 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-04-23T18:37:20.919 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-04-23T18:37:20.919 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-04-23T18:37:20.919 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-04-23T18:37:20.920 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-04-23T18:37:20.920 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-04-23T18:37:20.920 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-04-23T18:37:20.920 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-04-23T18:37:20.920 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-04-23T18:37:20.920 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-04-23T18:37:20.920 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-04-23T18:37:20.920 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-04-23T18:37:20.920 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-04-23T18:37:20.921 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-04-23T18:37:20.921 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-04-23T18:37:20.921 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-04-23T18:37:20.921 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-04-23T18:37:20.921 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-04-23T18:37:20.921 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-04-23T18:37:20.921 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-04-23T18:37:20.922 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-04-23T18:37:20.922 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-04-23T18:37:20.922 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-04-23T18:37:20.922 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-04-23T18:37:20.922 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-04-23T18:37:20.922 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-04-23T18:37:20.923 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-04-23T18:37:20.923 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-04-23T18:37:20.923 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-04-23T18:37:20.923 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-04-23T18:37:20.923 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-04-23T18:37:20.923 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-04-23T18:37:20.923 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-04-23T18:37:20.924 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-04-23T18:37:20.924 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-04-23T18:37:20.924 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-04-23T18:37:20.924 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-04-23T18:37:20.924 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-04-23T18:37:20.924 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-04-23T18:37:20.925 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-04-23T18:37:20.925 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-04-23T18:37:20.925 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-04-23T18:37:20.925 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-04-23T18:37:20.925 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-04-23T18:37:20.925 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-04-23T18:37:20.925 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-04-23T18:37:20.926 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-04-23T18:37:20.926 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-04-23T18:37:20.926 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-04-23T18:37:20.926 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-04-23T18:37:20.926 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-04-23T18:37:20.926 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-04-23T18:37:20.927 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-04-23T18:37:20.927 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-04-23T18:37:20.927 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-04-23T18:37:20.927 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-04-23T18:37:20.927 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-04-23T18:37:20.927 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-04-23T18:37:20.927 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-04-23T18:37:20.928 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-04-23T18:37:20.928 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-04-23T18:37:20.928 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-04-23T18:37:20.928 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-04-23T18:37:20.929 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-04-23T18:37:20.929 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-04-23T18:37:20.929 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-04-23T18:37:20.929 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-04-23T18:37:20.929 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-04-23T18:37:20.929 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-04-23T18:37:20.929 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-04-23T18:37:20.929 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-04-23T18:37:20.930 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-04-23T18:37:20.930 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-04-23T18:37:20.930 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-04-23T18:37:20.930 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-04-23T18:37:20.930 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-04-23T18:37:20.930 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-04-23T18:37:20.930 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-04-23T18:37:20.930 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-04-23T18:37:20.931 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-04-23T18:37:20.931 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-04-23T18:37:20.931 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-04-23T18:37:20.931 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-04-23T18:37:21.389 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-04-23T18:37:21.389 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-04-23T18:37:21.389 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-04-23T18:37:21.389 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-04-23T18:37:21.389 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-04-23T18:37:21.390 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-04-23T18:37:21.390 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-04-23T18:37:21.390 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-04-23T18:37:21.390 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-04-23T18:37:21.390 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-04-23T18:37:21.390 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-04-23T18:37:21.390 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-04-23T18:37:21.390 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-04-23T18:37:21.390 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-04-23T18:37:21.390 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-04-23T18:37:21.390 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-04-23T18:37:21.391 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-04-23T18:37:21.392 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-04-23T18:37:21.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-04-23T18:37:21.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-04-23T18:37:21.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-04-23T18:37:21.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-04-23T18:37:21.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-04-23T18:37:21.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-04-23T18:37:21.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-04-23T18:37:21.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-04-23T18:37:21.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-04-23T18:37:21.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-04-23T18:37:21.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-04-23T18:37:21.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:21 smithi148 ceph-mon[32311]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:37:21.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:21 smithi157 ceph-mon[32580]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:37:21.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:21 smithi132 ceph-mon[29178]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:37:21.761 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-04-23T18:37:21.761 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-04-23T18:37:21.761 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-04-23T18:37:21.762 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-04-23T18:37:21.762 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-04-23T18:37:21.762 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-04-23T18:37:21.762 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-04-23T18:37:21.762 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-04-23T18:37:21.762 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-04-23T18:37:21.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-04-23T18:37:21.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-04-23T18:37:21.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-04-23T18:37:21.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-04-23T18:37:21.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-04-23T18:37:21.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-04-23T18:37:21.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-04-23T18:37:21.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-04-23T18:37:21.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-04-23T18:37:21.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-04-23T18:37:21.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-04-23T18:37:21.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-04-23T18:37:21.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-04-23T18:37:21.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-04-23T18:37:21.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-04-23T18:37:21.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-04-23T18:37:21.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-04-23T18:37:21.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-04-23T18:37:21.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-04-23T18:37:21.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-04-23T18:37:21.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-04-23T18:37:21.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-04-23T18:37:21.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-04-23T18:37:21.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-04-23T18:37:21.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-04-23T18:37:21.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-04-23T18:37:21.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-04-23T18:37:21.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-04-23T18:37:21.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-04-23T18:37:21.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-04-23T18:37:21.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-04-23T18:37:21.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-04-23T18:37:21.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-04-23T18:37:21.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-04-23T18:37:21.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-04-23T18:37:21.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-04-23T18:37:21.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-04-23T18:37:21.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-04-23T18:37:21.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-04-23T18:37:21.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-04-23T18:37:21.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-04-23T18:37:21.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-04-23T18:37:21.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-04-23T18:37:21.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-04-23T18:37:21.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-04-23T18:37:21.770 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-04-23T18:37:21.770 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-04-23T18:37:21.770 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-04-23T18:37:21.770 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-04-23T18:37:21.770 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-04-23T18:37:21.770 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-04-23T18:37:21.771 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-04-23T18:37:21.771 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-04-23T18:37:21.771 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-04-23T18:37:21.771 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-04-23T18:37:21.771 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-04-23T18:37:21.771 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-04-23T18:37:21.772 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-04-23T18:37:21.772 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-04-23T18:37:21.772 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-04-23T18:37:21.772 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-04-23T18:37:21.772 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-04-23T18:37:21.772 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-04-23T18:37:21.773 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-04-23T18:37:21.773 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-04-23T18:37:21.773 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-04-23T18:37:21.773 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-04-23T18:37:21.773 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-04-23T18:37:21.773 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-04-23T18:37:21.775 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-04-23T18:37:21.775 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-04-23T18:37:21.775 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-04-23T18:37:21.775 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-04-23T18:37:21.775 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-04-23T18:37:21.775 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-04-23T18:37:21.775 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-04-23T18:37:21.775 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-04-23T18:37:22.361 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-04-23T18:37:22.361 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-04-23T18:37:22.361 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-04-23T18:37:22.361 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-04-23T18:37:22.361 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-04-23T18:37:22.361 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-04-23T18:37:22.362 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-04-23T18:37:22.362 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-04-23T18:37:22.362 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-04-23T18:37:22.362 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-04-23T18:37:22.362 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-04-23T18:37:22.362 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-04-23T18:37:22.363 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-04-23T18:37:22.363 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-04-23T18:37:22.363 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-04-23T18:37:22.363 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-04-23T18:37:22.363 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-04-23T18:37:22.363 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-04-23T18:37:22.363 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-04-23T18:37:22.363 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-04-23T18:37:22.363 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-04-23T18:37:22.363 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-04-23T18:37:22.364 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-04-23T18:37:22.364 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-04-23T18:37:22.364 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-04-23T18:37:22.364 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-04-23T18:37:22.364 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-04-23T18:37:22.364 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-04-23T18:37:22.364 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-04-23T18:37:22.364 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-04-23T18:37:22.364 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-04-23T18:37:22.364 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-04-23T18:37:22.364 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-04-23T18:37:22.365 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-04-23T18:37:22.365 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-04-23T18:37:22.365 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-04-23T18:37:22.365 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-04-23T18:37:22.365 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-04-23T18:37:22.365 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-04-23T18:37:22.365 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-04-23T18:37:22.365 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-04-23T18:37:22.365 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-04-23T18:37:22.365 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-04-23T18:37:22.366 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-04-23T18:37:22.366 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-04-23T18:37:22.366 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-04-23T18:37:22.366 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-04-23T18:37:22.366 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-04-23T18:37:22.366 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-04-23T18:37:22.366 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-04-23T18:37:22.367 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-04-23T18:37:22.367 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-04-23T18:37:22.367 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-04-23T18:37:22.367 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-04-23T18:37:22.367 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-04-23T18:37:22.367 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-04-23T18:37:22.367 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-04-23T18:37:22.367 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-04-23T18:37:22.368 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-04-23T18:37:22.368 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-04-23T18:37:22.368 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-04-23T18:37:22.368 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-04-23T18:37:22.368 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-04-23T18:37:22.368 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-04-23T18:37:22.368 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-04-23T18:37:22.368 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-04-23T18:37:22.368 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-04-23T18:37:22.369 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-04-23T18:37:22.369 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-04-23T18:37:22.369 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-04-23T18:37:22.369 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-04-23T18:37:22.370 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-04-23T18:37:22.370 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-04-23T18:37:22.370 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-04-23T18:37:22.370 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-04-23T18:37:22.370 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-04-23T18:37:22.370 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-04-23T18:37:22.370 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-04-23T18:37:22.371 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-04-23T18:37:22.371 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-04-23T18:37:22.371 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-04-23T18:37:22.371 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-04-23T18:37:22.371 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-04-23T18:37:22.371 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-04-23T18:37:22.525 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:22 smithi148 ceph-mon[32311]: pgmap v42: 97 pgs: 97 active+clean; 2.0 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 45 KiB/s wr, 2.71k op/s 2024-04-23T18:37:22.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:22 smithi157 ceph-mon[32580]: pgmap v42: 97 pgs: 97 active+clean; 2.0 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 45 KiB/s wr, 2.71k op/s 2024-04-23T18:37:22.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:22 smithi132 ceph-mon[29178]: pgmap v42: 97 pgs: 97 active+clean; 2.0 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 45 KiB/s wr, 2.71k op/s 2024-04-23T18:37:23.026 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-04-23T18:37:23.026 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-04-23T18:37:23.026 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-04-23T18:37:23.026 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-04-23T18:37:23.026 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-04-23T18:37:23.027 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-04-23T18:37:23.027 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-04-23T18:37:23.027 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-04-23T18:37:23.027 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-04-23T18:37:23.027 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-04-23T18:37:23.027 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-04-23T18:37:23.027 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-04-23T18:37:23.028 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-04-23T18:37:23.028 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-04-23T18:37:23.028 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-04-23T18:37:23.028 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-04-23T18:37:23.028 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-04-23T18:37:23.028 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-04-23T18:37:23.028 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-04-23T18:37:23.029 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-04-23T18:37:23.029 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-04-23T18:37:23.029 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-04-23T18:37:23.029 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-04-23T18:37:23.029 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-04-23T18:37:23.029 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-04-23T18:37:23.029 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-04-23T18:37:23.029 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-04-23T18:37:23.029 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-04-23T18:37:23.029 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-04-23T18:37:23.030 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-04-23T18:37:23.030 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-04-23T18:37:23.030 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-04-23T18:37:23.030 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-04-23T18:37:23.030 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-04-23T18:37:23.030 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-04-23T18:37:23.030 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-04-23T18:37:23.030 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-04-23T18:37:23.030 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-04-23T18:37:23.030 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-04-23T18:37:23.030 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-04-23T18:37:23.031 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-04-23T18:37:23.031 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-04-23T18:37:23.031 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-04-23T18:37:23.031 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-04-23T18:37:23.031 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-04-23T18:37:23.031 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-04-23T18:37:23.031 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-04-23T18:37:23.031 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-04-23T18:37:23.031 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-04-23T18:37:23.031 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-04-23T18:37:23.031 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-04-23T18:37:23.032 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-04-23T18:37:23.032 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-04-23T18:37:23.032 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-04-23T18:37:23.032 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-04-23T18:37:23.032 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-04-23T18:37:23.032 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-04-23T18:37:23.032 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-04-23T18:37:23.032 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-04-23T18:37:23.033 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-04-23T18:37:23.033 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-04-23T18:37:23.033 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-04-23T18:37:23.033 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-04-23T18:37:23.033 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-04-23T18:37:23.033 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-04-23T18:37:23.034 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-04-23T18:37:23.034 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-04-23T18:37:23.034 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-04-23T18:37:23.034 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-04-23T18:37:23.034 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-04-23T18:37:23.034 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-04-23T18:37:23.034 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-04-23T18:37:23.034 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-04-23T18:37:23.035 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-04-23T18:37:23.035 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-04-23T18:37:23.035 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-04-23T18:37:23.035 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-04-23T18:37:23.035 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-04-23T18:37:23.035 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-04-23T18:37:23.035 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-04-23T18:37:23.036 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-04-23T18:37:23.036 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-04-23T18:37:23.339 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-04-23T18:37:23.339 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-04-23T18:37:23.339 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-04-23T18:37:23.339 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-04-23T18:37:23.340 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-04-23T18:37:23.340 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-04-23T18:37:23.340 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-04-23T18:37:23.340 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-04-23T18:37:23.340 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-04-23T18:37:23.340 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-04-23T18:37:23.340 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-04-23T18:37:23.340 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-04-23T18:37:23.340 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-04-23T18:37:23.340 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-04-23T18:37:23.340 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-04-23T18:37:23.341 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-04-23T18:37:23.342 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-04-23T18:37:23.343 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-04-23T18:37:23.344 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-04-23T18:37:23.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-04-23T18:37:23.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-04-23T18:37:23.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-04-23T18:37:23.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-04-23T18:37:23.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-04-23T18:37:23.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-04-23T18:37:23.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-04-23T18:37:23.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-04-23T18:37:23.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-04-23T18:37:23.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-04-23T18:37:23.724 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-04-23T18:37:23.724 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-04-23T18:37:23.724 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-04-23T18:37:23.724 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-04-23T18:37:23.724 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-04-23T18:37:23.724 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-04-23T18:37:23.724 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-04-23T18:37:23.725 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-04-23T18:37:23.725 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-04-23T18:37:23.725 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-04-23T18:37:23.725 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-04-23T18:37:23.725 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-04-23T18:37:23.725 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-04-23T18:37:23.725 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-04-23T18:37:23.725 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-04-23T18:37:23.725 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-04-23T18:37:23.725 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-04-23T18:37:23.725 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-04-23T18:37:23.725 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-04-23T18:37:23.726 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-04-23T18:37:23.726 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-04-23T18:37:23.726 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-04-23T18:37:23.726 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-04-23T18:37:23.726 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-04-23T18:37:23.726 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-04-23T18:37:23.726 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-04-23T18:37:23.726 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-04-23T18:37:23.726 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-04-23T18:37:23.726 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-04-23T18:37:23.726 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-04-23T18:37:23.726 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-04-23T18:37:23.727 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-04-23T18:37:23.727 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-04-23T18:37:23.727 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-04-23T18:37:23.727 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-04-23T18:37:23.727 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-04-23T18:37:23.727 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-04-23T18:37:23.727 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-04-23T18:37:23.727 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-04-23T18:37:23.727 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-04-23T18:37:23.727 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-04-23T18:37:23.727 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-04-23T18:37:23.727 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-04-23T18:37:23.728 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-04-23T18:37:23.728 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-04-23T18:37:23.728 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-04-23T18:37:23.728 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-04-23T18:37:23.728 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-04-23T18:37:23.729 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-04-23T18:37:23.729 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-04-23T18:37:23.729 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-04-23T18:37:23.729 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-04-23T18:37:23.730 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-04-23T18:37:23.730 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-04-23T18:37:23.730 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-04-23T18:37:23.730 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-04-23T18:37:23.730 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-04-23T18:37:23.730 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-04-23T18:37:23.730 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-04-23T18:37:23.730 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-04-23T18:37:23.730 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-04-23T18:37:23.730 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-04-23T18:37:23.731 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-04-23T18:37:23.732 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-04-23T18:37:23.732 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-04-23T18:37:23.732 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-04-23T18:37:23.732 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-04-23T18:37:23.732 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-04-23T18:37:23.732 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-04-23T18:37:23.732 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-04-23T18:37:23.732 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-04-23T18:37:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.301+0000 7f9cc7708640 1 -- 172.21.15.132:0/2107165416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc010c530 msgr2=0x7f9cc010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.301+0000 7f9cc7708640 1 --2- 172.21.15.132:0/2107165416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc010c530 0x7f9cc010e9b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009ca0 tx=0x7f9cb802a950 comp rx=0 tx=0).stop 2024-04-23T18:37:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.301+0000 7f9cc7708640 1 -- 172.21.15.132:0/2107165416 shutdown_connections 2024-04-23T18:37:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.301+0000 7f9cc7708640 1 --2- 172.21.15.132:0/2107165416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc010c530 0x7f9cc010e9b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.301+0000 7f9cc7708640 1 --2- 172.21.15.132:0/2107165416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cc0103a80 0x7f9cc010bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.301+0000 7f9cc7708640 1 --2- 172.21.15.132:0/2107165416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc0103160 0x7f9cc0103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.301+0000 7f9cc7708640 1 -- 172.21.15.132:0/2107165416 >> 172.21.15.132:0/2107165416 conn(0x7f9cc00fc920 msgr2=0x7f9cc00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.301+0000 7f9cc7708640 1 -- 172.21.15.132:0/2107165416 shutdown_connections 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.301+0000 7f9cc7708640 1 -- 172.21.15.132:0/2107165416 wait complete. 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc7708640 1 Processor -- start 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc7708640 1 -- start start 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc7708640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc0103160 0x7f9cc0198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc7708640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cc0103a80 0x7f9cc0198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc7708640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc010c530 0x7f9cc019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc7708640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01121a0 con 0x7f9cc010c530 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc7708640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0112020 con 0x7f9cc0103a80 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc7708640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0112320 con 0x7f9cc0103160 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc5ca5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc010c530 0x7f9cc019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc54a4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc0103160 0x7f9cc0198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc54a4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc0103160 0x7f9cc0198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.132:36922/0 (socket says 172.21.15.132:36922) 2024-04-23T18:37:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc54a4640 1 -- 172.21.15.132:0/3213573384 learned_addr learned my addr 172.21.15.132:0/3213573384 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cc0103a80 0x7f9cc0198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc5ca5640 1 -- 172.21.15.132:0/3213573384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc0103160 msgr2=0x7f9cc0198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc5ca5640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc0103160 0x7f9cc0198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc5ca5640 1 -- 172.21.15.132:0/3213573384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cc0103a80 msgr2=0x7f9cc0198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc5ca5640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cc0103a80 0x7f9cc0198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc5ca5640 1 -- 172.21.15.132:0/3213573384 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc019d670 con 0x7f9cc010c530 2024-04-23T18:37:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.302+0000 7f9cc5ca5640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc010c530 0x7f9cc019cf40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005cc0 tx=0x7f9cb8031f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.303+0000 7f9cb67fc640 1 -- 172.21.15.132:0/3213573384 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8034040 con 0x7f9cc010c530 2024-04-23T18:37:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.303+0000 7f9cb67fc640 1 -- 172.21.15.132:0/3213573384 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb80308f0 con 0x7f9cc010c530 2024-04-23T18:37:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.303+0000 7f9cc7708640 1 -- 172.21.15.132:0/3213573384 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc019d900 con 0x7f9cc010c530 2024-04-23T18:37:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.303+0000 7f9cb67fc640 1 -- 172.21.15.132:0/3213573384 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80044d0 con 0x7f9cc010c530 2024-04-23T18:37:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.303+0000 7f9cc7708640 1 -- 172.21.15.132:0/3213573384 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc01a51f0 con 0x7f9cc010c530 2024-04-23T18:37:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.305+0000 7f9cca424740 1 Processor -- start 2024-04-23T18:37:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.307+0000 7f9cb67fc640 1 -- 172.21.15.132:0/3213573384 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f9cb8037020 con 0x7f9cc010c530 2024-04-23T18:37:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.307+0000 7f9cca424740 1 -- start start 2024-04-23T18:37:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.307+0000 7f9cca424740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c207f1500 0x559c207f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.307+0000 7f9cca424740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1e20 0x559c208006d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.307+0000 7f9cca424740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c20800c10 0x559c20803000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.307+0000 7f9cca424740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x559c20756970 con 0x559c207f1e20 2024-04-23T18:37:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.307+0000 7f9cca424740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559c206f34a0 con 0x559c207f1500 2024-04-23T18:37:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.307+0000 7f9cca424740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559c20752930 con 0x559c20800c10 2024-04-23T18:37:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc54a4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1e20 0x559c208006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc54a4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1e20 0x559c208006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46694/0 (socket says 172.21.15.132:46694) 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc54a4640 1 -- 172.21.15.132:0/3776466272 learned_addr learned my addr 172.21.15.132:0/3776466272 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cb67fc640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9cac0a3f70 0x7f9cac0a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/3776466272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c20800c10 0x559c20803000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc5ca5640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9cac0a3f70 0x7f9cac0a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc54a4640 1 -- 172.21.15.132:0/3776466272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c20800c10 msgr2=0x559c20803000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/3776466272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c207f1500 0x559c207f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc54a4640 1 --2- 172.21.15.132:0/3776466272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c20800c10 0x559c20803000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc54a4640 1 -- 172.21.15.132:0/3776466272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c207f1500 msgr2=0x559c207f18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc54a4640 1 --2- 172.21.15.132:0/3776466272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c207f1500 0x559c207f18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc54a4640 1 -- 172.21.15.132:0/3776466272 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c20469210 con 0x559c207f1e20 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cb67fc640 1 -- 172.21.15.132:0/3213573384 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f9cb80f16f0 con 0x7f9cc010c530 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/3776466272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c207f1500 0x559c207f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc54a4640 1 --2- 172.21.15.132:0/3776466272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1e20 0x559c208006d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009e40 tx=0x7f9cbc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc2ff8757239945 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cc5ca5640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9cac0a3f70 0x7f9cac0a6430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80eed60 tx=0x7f9cb808ce30 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cab7fe640 1 -- 172.21.15.132:0/3776466272 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc01e070 con 0x559c207f1e20 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cab7fe640 1 -- 172.21.15.132:0/3776466272 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cbc0198c0 con 0x559c207f1e20 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.308+0000 7f9cab7fe640 1 -- 172.21.15.132:0/3776466272 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc00f850 con 0x559c207f1e20 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.309+0000 7f9cca424740 1 -- 172.21.15.132:0/3776466272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1e20 msgr2=0x559c208006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.309+0000 7f9cca424740 1 --2- 172.21.15.132:0/3776466272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1e20 0x559c208006d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009e40 tx=0x7f9cbc011810 comp rx=0 tx=0).stop 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.309+0000 7f9cca424740 1 -- 172.21.15.132:0/3776466272 shutdown_connections 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.309+0000 7f9cca424740 1 --2- 172.21.15.132:0/3776466272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c20800c10 0x559c20803000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.309+0000 7f9cca424740 1 --2- 172.21.15.132:0/3776466272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1e20 0x559c208006d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.309+0000 7f9cca424740 1 --2- 172.21.15.132:0/3776466272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c207f1500 0x559c207f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.309+0000 7f9cca424740 1 -- 172.21.15.132:0/3776466272 >> 172.21.15.132:0/3776466272 conn(0x559c207edb30 msgr2=0x559c207edf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.309+0000 7f9cca424740 1 -- 172.21.15.132:0/3776466272 shutdown_connections 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.309+0000 7f9cca424740 1 -- 172.21.15.132:0/3776466272 wait complete. 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cca424740 1 -- start start 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cca424740 1 Processor -- start 2024-04-23T18:37:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cca424740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1500 0x559c20800540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cca424740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c207f1e20 0x559c207fd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cca424740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c20800c10 0x559c207fdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cca424740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x559c20803840 con 0x559c20800c10 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cca424740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559c206f10b0 con 0x559c207f1e20 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cca424740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559c208039c0 con 0x559c207f1500 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cca424740 10 client.0 _ll_register_callbacks cb 0x559c206ded30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cc4ca3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c20800c10 0x559c207fdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cc54a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c207f1e20 0x559c207fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cc4ca3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c20800c10 0x559c207fdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46698/0 (socket says 172.21.15.132:46698) 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cc4ca3640 1 -- 172.21.15.132:0/4131755820 learned_addr learned my addr 172.21.15.132:0/4131755820 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.310+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/4131755820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1500 0x559c20800540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cc4ca3640 1 -- 172.21.15.132:0/4131755820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1500 msgr2=0x559c20800540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/4131755820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1500 0x559c20800540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cc4ca3640 1 -- 172.21.15.132:0/4131755820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c207f1e20 msgr2=0x559c207fd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/4131755820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c207f1e20 0x559c207fd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cc4ca3640 1 -- 172.21.15.132:0/4131755820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c20535620 con 0x559c20800c10 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/4131755820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c20800c10 0x559c207fdd70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000f3c0 tx=0x7f9cb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49200b988936876 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9ca8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9ca8ff9640 1 -- 172.21.15.132:0/4131755820 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0005220 con 0x559c20800c10 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9ca8ff9640 1 -- 172.21.15.132:0/4131755820 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f9cb00053c0 con 0x559c20800c10 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cca424740 1 -- 172.21.15.132:0/4131755820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cbc0198c0 con 0x559c20800c10 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9ca8ff9640 1 -- 172.21.15.132:0/4131755820 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9cb00055c0 con 0x559c20800c10 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cca424740 10 client.15876 fetch_fsmap learned FSMap version 27 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cca424740 1 -- 172.21.15.132:0/4131755820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c2080f770 con 0x559c20800c10 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9ca8ff9640 1 -- 172.21.15.132:0/4131755820 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cbc0198c0 con 0x559c20800c10 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9ca8ff9640 1 -- 172.21.15.132:0/4131755820 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f9cb0020710 con 0x559c20800c10 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cca424740 10 client.15876 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cca424740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cca424740 20 client.15876 populate_metadata read hostname 'smithi132' 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cca424740 1 --2- 172.21.15.132:0/4131755820 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x559c20810c80 0x559c20813090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cca424740 4 client.15876 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.311+0000 7f9cca424740 1 -- 172.21.15.132:0/4131755820 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c20764390 con 0x559c20810c80 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.312+0000 7f9cc54a4640 1 --2- 172.21.15.132:0/4131755820 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x559c20810c80 0x559c20813090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.312+0000 7f9cc54a4640 1 --2- 172.21.15.132:0/4131755820 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x559c20810c80 0x559c20813090 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66be12e876a02257 server_cookie=4a59b11d63909de4 in_seq=0 out_seq=0 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.312+0000 7f9ca8ff9640 10 client.15876 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9ca8ff9640 1 -- 172.21.15.132:0/4131755820 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c20764390 con 0x559c20810c80 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9ca8ff9640 10 client.15876 handle_command_reply: tid=0 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9cca424740 1 client.15876 shutdown 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9cca424740 1 -- 172.21.15.132:0/4131755820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c20800c10 msgr2=0x559c207fdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9cca424740 1 --2- 172.21.15.132:0/4131755820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c20800c10 0x559c207fdd70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000f3c0 tx=0x7f9cb00046c0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9cca424740 1 -- 172.21.15.132:0/4131755820 shutdown_connections 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9cca424740 1 --2- 172.21.15.132:0/4131755820 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x559c20810c80 0x559c20813090 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9cca424740 1 --2- 172.21.15.132:0/4131755820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c20800c10 0x559c207fdd70 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9ca8ff9640 0 client.15876 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9cca424740 1 --2- 172.21.15.132:0/4131755820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c207f1e20 0x559c207fd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9cca424740 1 --2- 172.21.15.132:0/4131755820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1500 0x559c20800540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9cca424740 1 -- 172.21.15.132:0/4131755820 >> 172.21.15.132:0/4131755820 conn(0x559c207edb30 msgr2=0x559c207f3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9cca424740 1 -- 172.21.15.132:0/4131755820 shutdown_connections 2024-04-23T18:37:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9cca424740 1 -- 172.21.15.132:0/4131755820 wait complete. 2024-04-23T18:37:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.313+0000 7f9cca424740 20 client.15876 trim_cache size 0 max 16384 2024-04-23T18:37:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.329+0000 7f9cca424740 1 Processor -- start 2024-04-23T18:37:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.329+0000 7f9cca424740 1 -- start start 2024-04-23T18:37:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.329+0000 7f9cca424740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 0x559c2080c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.329+0000 7f9cca424740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1e20 0x559c20800070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.329+0000 7f9cca424740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c20800c10 0x559c20818960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.329+0000 7f9cca424740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01ab930 con 0x559c207f1500 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.329+0000 7f9cca424740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01ad4a0 con 0x559c20800c10 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.329+0000 7f9cca424740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01ac000 con 0x559c207f1e20 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9cc4ca3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 0x559c2080c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9cc4ca3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 0x559c2080c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46700/0 (socket says 172.21.15.132:46700) 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9cc4ca3640 1 -- 172.21.15.132:0/3392771670 learned_addr learned my addr 172.21.15.132:0/3392771670 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9cc54a4640 1 --2- 172.21.15.132:0/3392771670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1e20 0x559c20800070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/3392771670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c20800c10 0x559c20818960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9cc4ca3640 1 -- 172.21.15.132:0/3392771670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1e20 msgr2=0x559c20800070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/3392771670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1e20 0x559c20800070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9cc4ca3640 1 -- 172.21.15.132:0/3392771670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c20800c10 msgr2=0x559c20818960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/3392771670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c20800c10 0x559c20818960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9cc4ca3640 1 -- 172.21.15.132:0/3392771670 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c207f2880 con 0x559c207f1500 2024-04-23T18:37:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9cc54a4640 1 --2- 172.21.15.132:0/3392771670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1e20 0x559c20800070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:37:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/3392771670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 0x559c2080c770 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0011d50 tx=0x7f9cb0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95ab7d0404c4336 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9caaffd640 1 -- 172.21.15.132:0/3392771670 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0020610 con 0x559c207f1500 2024-04-23T18:37:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9caaffd640 1 -- 172.21.15.132:0/3392771670 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb00207b0 con 0x559c207f1500 2024-04-23T18:37:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.330+0000 7f9caaffd640 1 -- 172.21.15.132:0/3392771670 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0028410 con 0x559c207f1500 2024-04-23T18:37:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.331+0000 7f9cca424740 1 -- 172.21.15.132:0/3392771670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 msgr2=0x559c2080c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.331+0000 7f9cca424740 1 --2- 172.21.15.132:0/3392771670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 0x559c2080c770 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0011d50 tx=0x7f9cb0001e70 comp rx=0 tx=0).stop 2024-04-23T18:37:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.331+0000 7f9cca424740 1 -- 172.21.15.132:0/3392771670 shutdown_connections 2024-04-23T18:37:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.331+0000 7f9cca424740 1 --2- 172.21.15.132:0/3392771670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1e20 0x559c20800070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.331+0000 7f9cca424740 1 --2- 172.21.15.132:0/3392771670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c20800c10 0x559c20818960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.331+0000 7f9cca424740 1 --2- 172.21.15.132:0/3392771670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 0x559c2080c770 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.331+0000 7f9cca424740 1 -- 172.21.15.132:0/3392771670 >> 172.21.15.132:0/3392771670 conn(0x559c207edb30 msgr2=0x559c207fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.331+0000 7f9cca424740 1 -- 172.21.15.132:0/3392771670 shutdown_connections 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.331+0000 7f9cca424740 1 -- 172.21.15.132:0/3392771670 wait complete. 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.331+0000 7f9cca424740 1 -- start start 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.331+0000 7f9cca424740 1 Processor -- start 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cca424740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 0x559c20808d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cca424740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1e20 0x559c20804220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cca424740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c20800c10 0x559c20804760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cca424740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01acda0 con 0x559c207f1500 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cca424740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01ac6d0 con 0x559c20800c10 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cca424740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01a9910 con 0x559c207f1e20 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cca424740 10 client.0 _ll_register_callbacks cb 0x7f9cb0011c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cc4ca3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 0x559c20808d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cc4ca3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 0x559c20808d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46706/0 (socket says 172.21.15.132:46706) 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cc4ca3640 1 -- 172.21.15.132:0/1240633488 learned_addr learned my addr 172.21.15.132:0/1240633488 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cc4ca3640 1 -- 172.21.15.132:0/1240633488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1e20 msgr2=0x559c20804220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/1240633488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1e20 0x559c20804220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cc4ca3640 1 -- 172.21.15.132:0/1240633488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c20800c10 msgr2=0x559c20804760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/1240633488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c20800c10 0x559c20804760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cc4ca3640 1 -- 172.21.15.132:0/1240633488 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c20804cd0 con 0x559c207f1500 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9cc4ca3640 1 --2- 172.21.15.132:0/1240633488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 0x559c20808d90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000f3c0 tx=0x7f9cb0009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6803e9c291c1cca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.332+0000 7f9ca8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9ca8ff9640 1 -- 172.21.15.132:0/1240633488 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb003a070 con 0x559c207f1500 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9ca8ff9640 1 -- 172.21.15.132:0/1240633488 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f9cb0029470 con 0x559c207f1500 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9cca424740 1 -- 172.21.15.132:0/1240633488 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb00207b0 con 0x559c207f1500 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9ca8ff9640 1 -- 172.21.15.132:0/1240633488 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9cb0029670 con 0x559c207f1500 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9cca424740 10 client.15882 fetch_fsmap learned FSMap version 27 2024-04-23T18:37:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9cca424740 1 -- 172.21.15.132:0/1240633488 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c20876f60 con 0x559c207f1500 2024-04-23T18:37:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9ca8ff9640 1 -- 172.21.15.132:0/1240633488 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb00207b0 con 0x559c207f1500 2024-04-23T18:37:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9ca8ff9640 1 -- 172.21.15.132:0/1240633488 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f9cb0003ca0 con 0x559c207f1500 2024-04-23T18:37:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9cca424740 10 client.15882 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:37:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9cca424740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9cca424740 20 client.15882 populate_metadata read hostname 'smithi132' 2024-04-23T18:37:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9cca424740 1 --2- 172.21.15.132:0/1240633488 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x559c20810c80 0x559c20879600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9cca424740 4 client.15882 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:37:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.333+0000 7f9cca424740 1 -- 172.21.15.132:0/1240633488 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c2075e200 con 0x559c20810c80 2024-04-23T18:37:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.334+0000 7f9cc54a4640 1 --2- 172.21.15.132:0/1240633488 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x559c20810c80 0x559c20879600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.335+0000 7f9ca8ff9640 10 client.15882 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.335+0000 7f9cc54a4640 1 --2- 172.21.15.132:0/1240633488 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x559c20810c80 0x559c20879600 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c9bfa3d96257584 server_cookie=aebac32813dca1fb in_seq=0 out_seq=0 2024-04-23T18:37:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.346+0000 7f9ca8ff9640 1 -- 172.21.15.132:0/1240633488 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c2075e200 con 0x559c20810c80 2024-04-23T18:37:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.346+0000 7f9ca8ff9640 10 client.15882 handle_command_reply: tid=0 2024-04-23T18:37:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.346+0000 7f9cca424740 1 client.15882 shutdown 2024-04-23T18:37:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cca424740 1 -- 172.21.15.132:0/1240633488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 msgr2=0x559c20808d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cca424740 1 --2- 172.21.15.132:0/1240633488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 0x559c20808d90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000f3c0 tx=0x7f9cb0009fd0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cca424740 1 -- 172.21.15.132:0/1240633488 shutdown_connections 2024-04-23T18:37:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cca424740 1 --2- 172.21.15.132:0/1240633488 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x559c20810c80 0x559c20879600 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cca424740 1 --2- 172.21.15.132:0/1240633488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559c207f1e20 0x559c20804220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9ca8ff9640 0 client.15882 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cca424740 1 --2- 172.21.15.132:0/1240633488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c20800c10 0x559c20804760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cca424740 1 --2- 172.21.15.132:0/1240633488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c207f1500 0x559c20808d90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cca424740 1 -- 172.21.15.132:0/1240633488 >> 172.21.15.132:0/1240633488 conn(0x559c207edb30 msgr2=0x559c207f8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cca424740 1 -- 172.21.15.132:0/1240633488 shutdown_connections 2024-04-23T18:37:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cca424740 1 -- 172.21.15.132:0/1240633488 wait complete. 2024-04-23T18:37:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cca424740 20 client.15882 trim_cache size 0 max 16384 2024-04-23T18:37:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-04-23T18:37:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-04-23T18:37:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-04-23T18:37:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-04-23T18:37:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cc7708640 1 -- 172.21.15.132:0/3213573384 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9cac0a3f70 msgr2=0x7f9cac0a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cc7708640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9cac0a3f70 0x7f9cac0a6430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80eed60 tx=0x7f9cb808ce30 comp rx=0 tx=0).stop 2024-04-23T18:37:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cc7708640 1 -- 172.21.15.132:0/3213573384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc010c530 msgr2=0x7f9cc019cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cc7708640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc010c530 0x7f9cc019cf40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005cc0 tx=0x7f9cb8031f40 comp rx=0 tx=0).stop 2024-04-23T18:37:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cc7708640 1 -- 172.21.15.132:0/3213573384 shutdown_connections 2024-04-23T18:37:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cc7708640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9cac0a3f70 0x7f9cac0a6430 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cc7708640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc010c530 0x7f9cc019cf40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cc7708640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cc0103a80 0x7f9cc0198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cc7708640 1 --2- 172.21.15.132:0/3213573384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc0103160 0x7f9cc0198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.347+0000 7f9cc7708640 1 -- 172.21.15.132:0/3213573384 >> 172.21.15.132:0/3213573384 conn(0x7f9cc00fc920 msgr2=0x7f9cc00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.348+0000 7f9cc7708640 1 -- 172.21.15.132:0/3213573384 shutdown_connections 2024-04-23T18:37:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-04-23T18:37:24.348+0000 7f9cc7708640 1 -- 172.21.15.132:0/3213573384 wait complete. 2024-04-23T18:37:24.352 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-04-23T18:37:24.352 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-04-23T18:37:24.352 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-04-23T18:37:24.352 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-04-23T18:37:24.352 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-04-23T18:37:24.352 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-04-23T18:37:24.352 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-04-23T18:37:24.352 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-04-23T18:37:24.352 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-04-23T18:37:24.353 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-04-23T18:37:24.353 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-04-23T18:37:24.353 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-04-23T18:37:24.353 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-04-23T18:37:24.353 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-04-23T18:37:24.353 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-04-23T18:37:24.353 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-04-23T18:37:24.353 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-04-23T18:37:24.353 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-04-23T18:37:24.353 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-04-23T18:37:24.354 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-04-23T18:37:24.355 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-04-23T18:37:24.356 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-04-23T18:37:24.357 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-04-23T18:37:24.358 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-04-23T18:37:24.358 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-04-23T18:37:24.358 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-04-23T18:37:24.358 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-04-23T18:37:24.358 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-04-23T18:37:24.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:24 smithi148 ceph-mon[32311]: pgmap v43: 97 pgs: 97 active+clean; 2.0 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 45 KiB/s wr, 2.71k op/s 2024-04-23T18:37:24.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:24 smithi157 ceph-mon[32580]: pgmap v43: 97 pgs: 97 active+clean; 2.0 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 45 KiB/s wr, 2.71k op/s 2024-04-23T18:37:24.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:24 smithi132 ceph-mon[29178]: pgmap v43: 97 pgs: 97 active+clean; 2.0 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 45 KiB/s wr, 2.71k op/s 2024-04-23T18:37:24.957 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-04-23T18:37:24.957 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-04-23T18:37:24.957 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-04-23T18:37:24.957 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-04-23T18:37:24.957 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-04-23T18:37:24.958 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-04-23T18:37:24.959 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-04-23T18:37:24.960 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-04-23T18:37:24.961 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-04-23T18:37:24.961 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-04-23T18:37:24.961 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-04-23T18:37:24.961 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-04-23T18:37:24.961 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-04-23T18:37:24.961 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-04-23T18:37:24.961 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-04-23T18:37:24.961 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-04-23T18:37:24.961 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-04-23T18:37:24.961 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-04-23T18:37:24.962 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-04-23T18:37:24.962 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-04-23T18:37:24.962 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-04-23T18:37:24.962 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-04-23T18:37:24.962 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-04-23T18:37:24.962 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-04-23T18:37:24.962 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-04-23T18:37:24.962 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-04-23T18:37:24.962 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-04-23T18:37:24.962 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-04-23T18:37:24.962 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-04-23T18:37:24.962 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-04-23T18:37:24.963 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-04-23T18:37:24.963 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-04-23T18:37:24.963 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-04-23T18:37:24.963 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-04-23T18:37:24.963 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-04-23T18:37:24.963 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-04-23T18:37:24.963 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-04-23T18:37:24.963 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-04-23T18:37:24.963 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-04-23T18:37:24.963 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-04-23T18:37:24.963 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-04-23T18:37:24.963 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-04-23T18:37:24.964 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-04-23T18:37:25.291 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-04-23T18:37:25.292 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-04-23T18:37:25.292 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-04-23T18:37:25.292 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-04-23T18:37:25.292 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-04-23T18:37:25.292 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-04-23T18:37:25.292 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-04-23T18:37:25.292 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-04-23T18:37:25.292 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-04-23T18:37:25.293 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-04-23T18:37:25.293 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-04-23T18:37:25.293 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-04-23T18:37:25.293 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-04-23T18:37:25.293 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-04-23T18:37:25.293 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-04-23T18:37:25.293 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-04-23T18:37:25.293 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-04-23T18:37:25.293 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-04-23T18:37:25.293 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-04-23T18:37:25.293 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-04-23T18:37:25.294 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-04-23T18:37:25.294 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-04-23T18:37:25.294 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-04-23T18:37:25.294 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-04-23T18:37:25.294 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-04-23T18:37:25.294 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-04-23T18:37:25.294 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-04-23T18:37:25.294 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-04-23T18:37:25.295 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-04-23T18:37:25.295 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-04-23T18:37:25.295 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-04-23T18:37:25.295 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-04-23T18:37:25.295 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-04-23T18:37:25.295 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-04-23T18:37:25.295 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-04-23T18:37:25.295 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-04-23T18:37:25.295 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-04-23T18:37:25.295 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-04-23T18:37:25.295 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-04-23T18:37:25.295 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-04-23T18:37:25.296 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-04-23T18:37:25.296 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-04-23T18:37:25.296 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-04-23T18:37:25.296 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-04-23T18:37:25.296 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-04-23T18:37:25.296 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-04-23T18:37:25.296 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-04-23T18:37:25.296 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-04-23T18:37:25.297 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-04-23T18:37:25.297 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-04-23T18:37:25.297 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-04-23T18:37:25.297 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-04-23T18:37:25.297 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-04-23T18:37:25.297 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-04-23T18:37:25.297 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-04-23T18:37:25.297 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-04-23T18:37:25.297 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-04-23T18:37:25.297 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-04-23T18:37:25.298 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-04-23T18:37:25.298 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-04-23T18:37:25.298 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-04-23T18:37:25.298 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-04-23T18:37:25.298 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-04-23T18:37:25.298 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-04-23T18:37:25.298 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-04-23T18:37:25.298 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-04-23T18:37:25.299 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-04-23T18:37:25.299 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-04-23T18:37:25.299 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-04-23T18:37:25.299 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-04-23T18:37:25.299 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-04-23T18:37:25.299 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-04-23T18:37:25.299 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-04-23T18:37:25.299 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-04-23T18:37:25.299 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-04-23T18:37:25.300 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-04-23T18:37:25.300 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-04-23T18:37:25.300 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-04-23T18:37:25.300 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-04-23T18:37:25.300 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-04-23T18:37:25.300 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-04-23T18:37:25.751 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-04-23T18:37:25.751 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-04-23T18:37:25.751 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-04-23T18:37:25.751 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-04-23T18:37:25.751 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-04-23T18:37:25.751 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-04-23T18:37:25.751 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-04-23T18:37:25.751 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-04-23T18:37:25.751 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-04-23T18:37:25.752 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-04-23T18:37:25.753 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-04-23T18:37:25.754 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-04-23T18:37:25.755 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-04-23T18:37:25.756 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-04-23T18:37:26.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-04-23T18:37:26.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-04-23T18:37:26.345 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-04-23T18:37:26.346 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-04-23T18:37:26.347 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-04-23T18:37:26.347 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-04-23T18:37:26.347 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-04-23T18:37:26.347 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-04-23T18:37:26.347 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-04-23T18:37:26.347 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-04-23T18:37:26.347 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-04-23T18:37:26.347 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-04-23T18:37:26.347 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-04-23T18:37:26.347 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-04-23T18:37:26.347 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-04-23T18:37:26.347 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-04-23T18:37:26.348 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-04-23T18:37:26.349 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-04-23T18:37:26.350 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-04-23T18:37:26.350 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-04-23T18:37:26.350 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-04-23T18:37:26.350 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-04-23T18:37:26.350 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-04-23T18:37:26.350 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-04-23T18:37:26.350 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-04-23T18:37:26.350 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-04-23T18:37:26.350 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-04-23T18:37:26.350 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-04-23T18:37:26.350 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-04-23T18:37:26.350 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-04-23T18:37:26.351 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-04-23T18:37:26.351 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-04-23T18:37:26.351 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-04-23T18:37:26.351 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-04-23T18:37:26.351 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-04-23T18:37:26.351 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-04-23T18:37:26.351 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-04-23T18:37:26.351 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-04-23T18:37:26.351 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-04-23T18:37:26.412 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:26 smithi132 ceph-mon[29178]: pgmap v44: 97 pgs: 97 active+clean; 220 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 1.1 MiB/s wr, 2.99k op/s 2024-04-23T18:37:26.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:26 smithi148 ceph-mon[32311]: pgmap v44: 97 pgs: 97 active+clean; 220 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 1.1 MiB/s wr, 2.99k op/s 2024-04-23T18:37:26.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:26 smithi157 ceph-mon[32580]: pgmap v44: 97 pgs: 97 active+clean; 220 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 1.1 MiB/s wr, 2.99k op/s 2024-04-23T18:37:26.847 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-04-23T18:37:26.847 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-04-23T18:37:26.847 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-04-23T18:37:26.848 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-04-23T18:37:26.849 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-04-23T18:37:26.850 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-04-23T18:37:26.851 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-04-23T18:37:26.852 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-04-23T18:37:26.853 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-04-23T18:37:26.853 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-04-23T18:37:26.853 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-04-23T18:37:26.853 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-04-23T18:37:26.853 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-04-23T18:37:26.853 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-04-23T18:37:26.853 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-04-23T18:37:27.206 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-04-23T18:37:27.206 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-04-23T18:37:27.206 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-04-23T18:37:27.206 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-04-23T18:37:27.207 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-04-23T18:37:27.208 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-04-23T18:37:27.209 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-04-23T18:37:27.210 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-04-23T18:37:27.211 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-04-23T18:37:27.212 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-04-23T18:37:27.212 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-04-23T18:37:27.212 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-04-23T18:37:27.739 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-04-23T18:37:27.739 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-04-23T18:37:27.739 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-04-23T18:37:27.740 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-04-23T18:37:27.741 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-04-23T18:37:27.742 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-04-23T18:37:27.743 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-04-23T18:37:27.744 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-04-23T18:37:27.745 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-04-23T18:37:27.745 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-04-23T18:37:27.745 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-04-23T18:37:28.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-04-23T18:37:28.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-04-23T18:37:28.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-04-23T18:37:28.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-04-23T18:37:28.393 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-04-23T18:37:28.394 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-04-23T18:37:28.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-04-23T18:37:28.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-04-23T18:37:28.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-04-23T18:37:28.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-04-23T18:37:28.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-04-23T18:37:28.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-04-23T18:37:28.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-04-23T18:37:28.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-04-23T18:37:28.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-04-23T18:37:28.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-04-23T18:37:28.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-04-23T18:37:28.395 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-04-23T18:37:28.396 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-04-23T18:37:28.397 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-04-23T18:37:28.398 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-04-23T18:37:28.398 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-04-23T18:37:28.398 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-04-23T18:37:28.398 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-04-23T18:37:28.398 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-04-23T18:37:28.398 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-04-23T18:37:28.398 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-04-23T18:37:28.398 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-04-23T18:37:28.398 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-04-23T18:37:28.398 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-04-23T18:37:28.398 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-04-23T18:37:28.399 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-04-23T18:37:28.399 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-04-23T18:37:28.399 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-04-23T18:37:28.399 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-04-23T18:37:28.399 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-04-23T18:37:28.399 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-04-23T18:37:28.399 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-04-23T18:37:28.399 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-04-23T18:37:28.399 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-04-23T18:37:28.399 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-04-23T18:37:28.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:28 smithi148 ceph-mon[32311]: pgmap v45: 97 pgs: 97 active+clean; 220 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 1.0 MiB/s wr, 1.38k op/s 2024-04-23T18:37:28.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:28 smithi157 ceph-mon[32580]: pgmap v45: 97 pgs: 97 active+clean; 220 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 1.0 MiB/s wr, 1.38k op/s 2024-04-23T18:37:28.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:28 smithi132 ceph-mon[29178]: pgmap v45: 97 pgs: 97 active+clean; 220 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 1.0 MiB/s wr, 1.38k op/s 2024-04-23T18:37:28.790 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-04-23T18:37:28.790 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-04-23T18:37:28.790 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-04-23T18:37:28.790 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-04-23T18:37:28.790 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-04-23T18:37:28.790 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-04-23T18:37:28.790 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-04-23T18:37:28.790 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-04-23T18:37:28.790 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-04-23T18:37:28.790 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-04-23T18:37:28.790 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-04-23T18:37:28.791 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-04-23T18:37:28.792 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-04-23T18:37:28.793 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-04-23T18:37:28.794 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-04-23T18:37:28.795 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-04-23T18:37:28.795 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-04-23T18:37:28.795 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-04-23T18:37:28.795 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-04-23T18:37:28.795 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-04-23T18:37:28.795 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-04-23T18:37:28.795 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-04-23T18:37:28.795 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-04-23T18:37:28.795 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-04-23T18:37:28.795 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-04-23T18:37:28.795 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-04-23T18:37:28.795 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-04-23T18:37:29.104 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-04-23T18:37:29.104 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-04-23T18:37:29.104 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-04-23T18:37:29.104 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-04-23T18:37:29.104 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-04-23T18:37:29.104 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-04-23T18:37:29.104 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-04-23T18:37:29.104 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-04-23T18:37:29.105 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-04-23T18:37:29.105 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-04-23T18:37:29.105 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-04-23T18:37:29.105 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-04-23T18:37:29.105 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-04-23T18:37:29.105 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-04-23T18:37:29.105 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-04-23T18:37:29.105 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-04-23T18:37:29.105 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-04-23T18:37:29.105 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-04-23T18:37:29.105 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-04-23T18:37:29.105 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-04-23T18:37:29.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-04-23T18:37:29.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-04-23T18:37:29.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-04-23T18:37:29.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-04-23T18:37:29.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-04-23T18:37:29.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-04-23T18:37:29.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-04-23T18:37:29.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-04-23T18:37:29.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-04-23T18:37:29.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-04-23T18:37:29.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-04-23T18:37:29.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-04-23T18:37:29.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-04-23T18:37:29.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-04-23T18:37:29.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-04-23T18:37:29.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-04-23T18:37:29.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-04-23T18:37:29.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-04-23T18:37:29.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-04-23T18:37:29.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-04-23T18:37:29.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-04-23T18:37:29.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-04-23T18:37:29.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-04-23T18:37:29.110 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-04-23T18:37:29.110 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-04-23T18:37:29.110 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-04-23T18:37:29.110 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-04-23T18:37:29.110 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-04-23T18:37:29.110 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-04-23T18:37:29.111 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-04-23T18:37:29.111 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-04-23T18:37:29.111 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-04-23T18:37:29.111 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-04-23T18:37:29.111 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-04-23T18:37:29.111 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-04-23T18:37:29.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-04-23T18:37:29.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-04-23T18:37:29.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-04-23T18:37:29.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-04-23T18:37:29.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-04-23T18:37:29.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-04-23T18:37:29.763 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-04-23T18:37:29.764 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-04-23T18:37:29.765 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-04-23T18:37:29.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-04-23T18:37:29.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-04-23T18:37:29.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-04-23T18:37:29.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-04-23T18:37:29.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-04-23T18:37:29.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-04-23T18:37:29.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-04-23T18:37:29.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-04-23T18:37:29.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-04-23T18:37:29.766 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-04-23T18:37:29.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-04-23T18:37:29.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-04-23T18:37:29.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-04-23T18:37:29.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-04-23T18:37:29.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-04-23T18:37:29.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-04-23T18:37:29.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-04-23T18:37:29.767 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-04-23T18:37:29.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-04-23T18:37:29.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-04-23T18:37:29.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-04-23T18:37:29.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-04-23T18:37:29.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-04-23T18:37:29.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-04-23T18:37:29.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-04-23T18:37:29.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-04-23T18:37:29.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-04-23T18:37:29.768 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-04-23T18:37:29.769 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-04-23T18:37:29.770 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-04-23T18:37:29.770 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-04-23T18:37:30.309 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-04-23T18:37:30.309 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-04-23T18:37:30.309 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-04-23T18:37:30.310 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-04-23T18:37:30.311 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-04-23T18:37:30.312 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-04-23T18:37:30.313 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-04-23T18:37:30.314 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-04-23T18:37:30.315 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-04-23T18:37:30.315 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-04-23T18:37:30.315 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-04-23T18:37:30.315 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-04-23T18:37:30.315 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-04-23T18:37:30.315 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-04-23T18:37:30.315 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-04-23T18:37:30.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:30 smithi148 ceph-mon[32311]: pgmap v46: 97 pgs: 97 active+clean; 287 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s rd, 1.5 MiB/s wr, 1.60k op/s 2024-04-23T18:37:30.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:30 smithi157 ceph-mon[32580]: pgmap v46: 97 pgs: 97 active+clean; 287 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s rd, 1.5 MiB/s wr, 1.60k op/s 2024-04-23T18:37:30.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:30 smithi132 ceph-mon[29178]: pgmap v46: 97 pgs: 97 active+clean; 287 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s rd, 1.5 MiB/s wr, 1.60k op/s 2024-04-23T18:37:30.653 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-04-23T18:37:30.653 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-04-23T18:37:30.653 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-04-23T18:37:30.653 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-04-23T18:37:30.653 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-04-23T18:37:30.654 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-04-23T18:37:30.655 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-04-23T18:37:30.656 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-04-23T18:37:30.657 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-04-23T18:37:30.658 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-04-23T18:37:30.658 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-04-23T18:37:30.658 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-04-23T18:37:30.658 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-04-23T18:37:30.658 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-04-23T18:37:30.658 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-04-23T18:37:30.658 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-04-23T18:37:30.658 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-04-23T18:37:30.658 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-04-23T18:37:30.658 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-04-23T18:37:30.658 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-04-23T18:37:30.658 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-04-23T18:37:30.659 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-04-23T18:37:30.659 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-04-23T18:37:30.659 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-04-23T18:37:30.659 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-04-23T18:37:31.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-04-23T18:37:31.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-04-23T18:37:31.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-04-23T18:37:31.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-04-23T18:37:31.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-04-23T18:37:31.083 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-04-23T18:37:31.084 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-04-23T18:37:31.085 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-04-23T18:37:31.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-04-23T18:37:31.699 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-04-23T18:37:31.699 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-04-23T18:37:31.699 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-04-23T18:37:31.699 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-04-23T18:37:31.699 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-04-23T18:37:31.699 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-04-23T18:37:31.700 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-04-23T18:37:31.701 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-04-23T18:37:31.702 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-04-23T18:37:31.703 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-04-23T18:37:31.704 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-04-23T18:37:31.704 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-04-23T18:37:31.704 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-04-23T18:37:31.704 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-04-23T18:37:31.704 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-04-23T18:37:31.704 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-04-23T18:37:31.704 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-04-23T18:37:31.704 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-04-23T18:37:31.704 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-04-23T18:37:31.704 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-04-23T18:37:31.704 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-04-23T18:37:31.704 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-04-23T18:37:32.226 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-04-23T18:37:32.226 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-04-23T18:37:32.226 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-04-23T18:37:32.226 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-04-23T18:37:32.226 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-04-23T18:37:32.226 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-04-23T18:37:32.226 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-04-23T18:37:32.226 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-04-23T18:37:32.226 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-04-23T18:37:32.227 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-04-23T18:37:32.228 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-04-23T18:37:32.229 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-04-23T18:37:32.230 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-04-23T18:37:32.231 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-04-23T18:37:32.231 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-04-23T18:37:32.231 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-04-23T18:37:32.231 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-04-23T18:37:32.231 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-04-23T18:37:32.231 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-04-23T18:37:32.231 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-04-23T18:37:32.231 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-04-23T18:37:32.231 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-04-23T18:37:32.231 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-04-23T18:37:32.231 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-04-23T18:37:32.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:32 smithi148 ceph-mon[32311]: pgmap v47: 97 pgs: 97 active+clean; 287 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 1.5 MiB/s wr, 500 op/s 2024-04-23T18:37:32.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:32 smithi157 ceph-mon[32580]: pgmap v47: 97 pgs: 97 active+clean; 287 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 1.5 MiB/s wr, 500 op/s 2024-04-23T18:37:32.555 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-04-23T18:37:32.556 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-04-23T18:37:32.557 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-04-23T18:37:32.558 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-04-23T18:37:32.559 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-04-23T18:37:32.560 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-04-23T18:37:32.560 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-04-23T18:37:32.560 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-04-23T18:37:32.560 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-04-23T18:37:32.560 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-04-23T18:37:32.560 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-04-23T18:37:32.560 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-04-23T18:37:32.561 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-04-23T18:37:32.561 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-04-23T18:37:32.561 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-04-23T18:37:32.561 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-04-23T18:37:32.561 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-04-23T18:37:32.561 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-04-23T18:37:32.561 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-04-23T18:37:32.561 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-04-23T18:37:32.561 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-04-23T18:37:32.561 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-04-23T18:37:32.561 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-04-23T18:37:32.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:32 smithi132 ceph-mon[29178]: pgmap v47: 97 pgs: 97 active+clean; 287 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 1.5 MiB/s wr, 500 op/s 2024-04-23T18:37:33.077 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-04-23T18:37:33.077 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-04-23T18:37:33.077 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-04-23T18:37:33.077 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-04-23T18:37:33.078 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-04-23T18:37:33.079 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-04-23T18:37:33.080 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-04-23T18:37:33.081 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-04-23T18:37:33.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-04-23T18:37:33.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-04-23T18:37:33.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-04-23T18:37:33.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-04-23T18:37:33.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-04-23T18:37:33.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-04-23T18:37:33.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-04-23T18:37:33.082 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-04-23T18:37:33.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-04-23T18:37:33.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-04-23T18:37:33.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-04-23T18:37:33.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-04-23T18:37:33.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-04-23T18:37:33.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-04-23T18:37:33.086 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-04-23T18:37:33.708 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-04-23T18:37:33.708 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-04-23T18:37:33.708 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-04-23T18:37:33.708 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-04-23T18:37:33.708 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-04-23T18:37:33.708 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-04-23T18:37:33.708 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-04-23T18:37:33.708 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-04-23T18:37:33.708 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-04-23T18:37:33.708 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-04-23T18:37:33.708 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-04-23T18:37:33.708 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-04-23T18:37:33.709 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-04-23T18:37:33.710 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-04-23T18:37:33.711 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-04-23T18:37:33.712 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-04-23T18:37:33.713 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-04-23T18:37:33.713 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-04-23T18:37:33.713 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-04-23T18:37:33.713 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-04-23T18:37:33.713 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-04-23T18:37:33.713 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-04-23T18:37:34.096 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-04-23T18:37:34.096 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-04-23T18:37:34.096 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-04-23T18:37:34.096 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-04-23T18:37:34.097 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-04-23T18:37:34.097 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-04-23T18:37:34.097 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-04-23T18:37:34.097 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-04-23T18:37:34.097 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-04-23T18:37:34.097 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-04-23T18:37:34.097 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-04-23T18:37:34.098 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-04-23T18:37:34.098 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-04-23T18:37:34.098 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-04-23T18:37:34.098 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-04-23T18:37:34.098 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-04-23T18:37:34.098 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-04-23T18:37:34.098 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-04-23T18:37:34.099 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-04-23T18:37:34.099 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-04-23T18:37:34.099 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-04-23T18:37:34.099 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-04-23T18:37:34.099 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-04-23T18:37:34.099 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-04-23T18:37:34.099 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-04-23T18:37:34.100 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-04-23T18:37:34.100 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-04-23T18:37:34.100 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-04-23T18:37:34.100 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-04-23T18:37:34.100 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-04-23T18:37:34.100 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-04-23T18:37:34.100 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-04-23T18:37:34.100 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-04-23T18:37:34.101 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-04-23T18:37:34.101 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-04-23T18:37:34.101 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-04-23T18:37:34.101 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-04-23T18:37:34.101 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-04-23T18:37:34.101 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-04-23T18:37:34.101 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-04-23T18:37:34.101 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-04-23T18:37:34.102 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-04-23T18:37:34.102 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-04-23T18:37:34.102 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-04-23T18:37:34.102 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-04-23T18:37:34.102 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-04-23T18:37:34.102 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-04-23T18:37:34.102 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-04-23T18:37:34.103 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-04-23T18:37:34.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-04-23T18:37:34.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-04-23T18:37:34.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-04-23T18:37:34.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-04-23T18:37:34.106 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-04-23T18:37:34.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-04-23T18:37:34.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-04-23T18:37:34.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-04-23T18:37:34.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-04-23T18:37:34.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-04-23T18:37:34.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-04-23T18:37:34.107 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-04-23T18:37:34.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-04-23T18:37:34.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-04-23T18:37:34.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-04-23T18:37:34.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-04-23T18:37:34.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-04-23T18:37:34.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-04-23T18:37:34.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-04-23T18:37:34.108 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-04-23T18:37:34.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-04-23T18:37:34.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-04-23T18:37:34.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-04-23T18:37:34.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-04-23T18:37:34.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-04-23T18:37:34.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-04-23T18:37:34.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-04-23T18:37:34.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-04-23T18:37:34.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-04-23T18:37:34.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-04-23T18:37:34.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-04-23T18:37:34.109 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-04-23T18:37:34.447 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-04-23T18:37:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-04-23T18:37:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-04-23T18:37:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-04-23T18:37:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-04-23T18:37:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-04-23T18:37:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-04-23T18:37:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-04-23T18:37:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-04-23T18:37:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-04-23T18:37:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-04-23T18:37:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-04-23T18:37:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-04-23T18:37:34.449 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-04-23T18:37:34.450 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-04-23T18:37:34.451 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-04-23T18:37:34.452 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-04-23T18:37:34.453 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-04-23T18:37:34.453 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-04-23T18:37:34.453 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-04-23T18:37:34.453 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-04-23T18:37:34.453 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-04-23T18:37:34.453 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-04-23T18:37:34.453 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-04-23T18:37:34.453 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-04-23T18:37:34.453 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-04-23T18:37:34.509 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-04-23T18:37:34.509 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-04-23T18:37:34.509 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-04-23T18:37:34.509 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-04-23T18:37:34.509 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-04-23T18:37:34.509 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-04-23T18:37:34.509 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-04-23T18:37:34.509 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-04-23T18:37:34.509 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-04-23T18:37:34.510 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-04-23T18:37:34.510 INFO:tasks.workunit.client.0.smithi132.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-04-23T18:37:34.510 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-04-23T18:37:34.510 INFO:tasks.workunit.client.0.smithi132.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-04-23T18:37:34.510 INFO:tasks.workunit.client.0.smithi132.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-04-23T18:37:34.514 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-23T18:37:34.514 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-23T18:37:34.515 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:34 smithi148 ceph-mon[32311]: pgmap v48: 97 pgs: 97 active+clean; 287 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 1.5 MiB/s wr, 500 op/s 2024-04-23T18:37:34.534 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:34 smithi132 ceph-mon[29178]: pgmap v48: 97 pgs: 97 active+clean; 287 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 1.5 MiB/s wr, 500 op/s 2024-04-23T18:37:34.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:34 smithi157 ceph-mon[32580]: pgmap v48: 97 pgs: 97 active+clean; 287 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 1.5 MiB/s wr, 500 op/s 2024-04-23T18:37:34.553 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-04-23T18:37:34.553 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-23T18:37:34.956 DEBUG:teuthology.parallel:result is None 2024-04-23T18:37:34.956 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-23T18:37:34.988 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-23T18:37:34.988 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-23T18:37:34.998 INFO:tasks.quiescer:joining Quiescers 2024-04-23T18:37:34.998 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-23T18:37:34.998 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:37:35.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.135+0000 7fe606be2640 1 -- 172.21.15.132:0/3885300384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6001031b0 msgr2=0x7fe600103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.135+0000 7fe606be2640 1 --2- 172.21.15.132:0/3885300384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6001031b0 0x7fe600103590 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009900 tx=0x7fe5f402a480 comp rx=0 tx=0).stop 2024-04-23T18:37:35.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.135+0000 7fe606be2640 1 -- 172.21.15.132:0/3885300384 shutdown_connections 2024-04-23T18:37:35.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.135+0000 7fe606be2640 1 --2- 172.21.15.132:0/3885300384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe600103ad0 0x7fe60010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.135+0000 7fe606be2640 1 --2- 172.21.15.132:0/3885300384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe60010c580 0x7fe60010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.135+0000 7fe606be2640 1 --2- 172.21.15.132:0/3885300384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6001031b0 0x7fe600103590 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.135+0000 7fe606be2640 1 -- 172.21.15.132:0/3885300384 >> 172.21.15.132:0/3885300384 conn(0x7fe6000771d0 msgr2=0x7fe6000775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.135+0000 7fe606be2640 1 -- 172.21.15.132:0/3885300384 shutdown_connections 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.135+0000 7fe606be2640 1 -- 172.21.15.132:0/3885300384 wait complete. 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe606be2640 1 Processor -- start 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe606be2640 1 -- start start 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe606be2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6001031b0 0x7fe60019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe606be2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe600103ad0 0x7fe60019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe606be2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe60010c580 0x7fe600198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe606be2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe6001121d0 con 0x7fe6001031b0 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe606be2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe600112050 con 0x7fe60010c580 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe606be2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe600112350 con 0x7fe600103ad0 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe60497e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6001031b0 0x7fe60019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe60497e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6001031b0 0x7fe60019e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41350/0 (socket says 172.21.15.132:41350) 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe60497e640 1 -- 172.21.15.132:0/306502811 learned_addr learned my addr 172.21.15.132:0/306502811 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe5fffff640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe600103ad0 0x7fe60019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe60517f640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe60010c580 0x7fe600198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe60497e640 1 -- 172.21.15.132:0/306502811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe600103ad0 msgr2=0x7fe60019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe60497e640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe600103ad0 0x7fe60019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe60497e640 1 -- 172.21.15.132:0/306502811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe60010c580 msgr2=0x7fe600198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe60497e640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe60010c580 0x7fe600198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe60497e640 1 -- 172.21.15.132:0/306502811 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe600198da0 con 0x7fe6001031b0 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.136+0000 7fe60497e640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6001031b0 0x7fe60019e470 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4005940 tx=0x7fe5f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.137+0000 7fe5fdffb640 1 -- 172.21.15.132:0/306502811 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f403f030 con 0x7fe6001031b0 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.137+0000 7fe5fdffb640 1 -- 172.21.15.132:0/306502811 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f4003cb0 con 0x7fe6001031b0 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.137+0000 7fe606be2640 1 -- 172.21.15.132:0/306502811 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe600199030 con 0x7fe6001031b0 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.137+0000 7fe5fdffb640 1 -- 172.21.15.132:0/306502811 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4035070 con 0x7fe6001031b0 2024-04-23T18:37:35.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.137+0000 7fe606be2640 1 -- 172.21.15.132:0/306502811 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6001a5330 con 0x7fe6001031b0 2024-04-23T18:37:35.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.138+0000 7fe606be2640 1 -- 172.21.15.132:0/306502811 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe60006add0 con 0x7fe6001031b0 2024-04-23T18:37:35.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.139+0000 7fe5fdffb640 1 -- 172.21.15.132:0/306502811 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fe5f4042030 con 0x7fe6001031b0 2024-04-23T18:37:35.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.139+0000 7fe5fdffb640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe5f00a3ed0 0x7fe5f00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.140+0000 7fe5fffff640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe5f00a3ed0 0x7fe5f00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.140+0000 7fe5fdffb640 1 -- 172.21.15.132:0/306502811 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fe5f40f15b0 con 0x7fe6001031b0 2024-04-23T18:37:35.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.140+0000 7fe5fffff640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe5f00a3ed0 0x7fe5f00a6390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe60010e4a0 tx=0x7fe5ec00b390 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:35.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.143+0000 7fe5fdffb640 1 -- 172.21.15.132:0/306502811 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5f40bda00 con 0x7fe6001031b0 2024-04-23T18:37:35.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.185+0000 7fe5fdffb640 1 -- 172.21.15.132:0/306502811 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fe5f40b9970 con 0x7fe6001031b0 2024-04-23T18:37:35.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.369+0000 7fe606be2640 1 -- 172.21.15.132:0/306502811 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6000691c0 con 0x7fe6001031b0 2024-04-23T18:37:35.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.369+0000 7fe5fdffb640 1 -- 172.21.15.132:0/306502811 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11947 (secure 0 0 0) 0x7fe5f4003e60 con 0x7fe6001031b0 2024-04-23T18:37:35.362 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:37:35.362 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:37:19.527407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24515,24491,24394,24460,24425]},"id":1}]} 2024-04-23T18:37:35.362 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 27 2024-04-23T18:37:35.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.371+0000 7fe606be2640 1 -- 172.21.15.132:0/306502811 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe5f00a3ed0 msgr2=0x7fe5f00a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.371+0000 7fe606be2640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe5f00a3ed0 0x7fe5f00a6390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe60010e4a0 tx=0x7fe5ec00b390 comp rx=0 tx=0).stop 2024-04-23T18:37:35.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.371+0000 7fe606be2640 1 -- 172.21.15.132:0/306502811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6001031b0 msgr2=0x7fe60019e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.371+0000 7fe606be2640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6001031b0 0x7fe60019e470 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4005940 tx=0x7fe5f40042f0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.372+0000 7fe606be2640 1 -- 172.21.15.132:0/306502811 shutdown_connections 2024-04-23T18:37:35.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.372+0000 7fe606be2640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe5f00a3ed0 0x7fe5f00a6390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.372+0000 7fe606be2640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe600103ad0 0x7fe60019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.372+0000 7fe606be2640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe60010c580 0x7fe600198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.372+0000 7fe606be2640 1 --2- 172.21.15.132:0/306502811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6001031b0 0x7fe60019e470 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.372+0000 7fe606be2640 1 -- 172.21.15.132:0/306502811 >> 172.21.15.132:0/306502811 conn(0x7fe6000771d0 msgr2=0x7fe6000fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:35.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.372+0000 7fe606be2640 1 -- 172.21.15.132:0/306502811 shutdown_connections 2024-04-23T18:37:35.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.372+0000 7fe606be2640 1 -- 172.21.15.132:0/306502811 wait complete. 2024-04-23T18:37:35.376 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24515 quiesce db --cancel --all' 2024-04-23T18:37:35.376 DEBUG:teuthology.orchestra.run.smithi132:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24515 quiesce db --cancel --all 2024-04-23T18:37:35.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.487+0000 7f080cbcd640 1 -- 172.21.15.132:0/1579150416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f080810c510 msgr2=0x7f080810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.487+0000 7f080cbcd640 1 --2- 172.21.15.132:0/1579150416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f080810c510 0x7f080810e9b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0800009ca0 tx=0x7f080002a9e0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.487+0000 7f080cbcd640 1 -- 172.21.15.132:0/1579150416 shutdown_connections 2024-04-23T18:37:35.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.487+0000 7f080cbcd640 1 --2- 172.21.15.132:0/1579150416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f080810c510 0x7f080810e9b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.487+0000 7f080cbcd640 1 --2- 172.21.15.132:0/1579150416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0808103af0 0x7f080810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.487+0000 7f080cbcd640 1 --2- 172.21.15.132:0/1579150416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08081031d0 0x7f08081035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.487+0000 7f080cbcd640 1 -- 172.21.15.132:0/1579150416 >> 172.21.15.132:0/1579150416 conn(0x7f0808077f50 msgr2=0x7f0808078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.487+0000 7f080cbcd640 1 -- 172.21.15.132:0/1579150416 shutdown_connections 2024-04-23T18:37:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.487+0000 7f080cbcd640 1 -- 172.21.15.132:0/1579150416 wait complete. 2024-04-23T18:37:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080cbcd640 1 Processor -- start 2024-04-23T18:37:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080cbcd640 1 -- start start 2024-04-23T18:37:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080cbcd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08081031d0 0x7f08081986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080cbcd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808103af0 0x7f0808198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080cbcd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f080810c510 0x7f080819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080cbcd640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0808112180 con 0x7f08081031d0 2024-04-23T18:37:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080cbcd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0808112000 con 0x7f080810c510 2024-04-23T18:37:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080cbcd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0808112300 con 0x7f0808103af0 2024-04-23T18:37:35.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08081031d0 0x7f08081986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08081031d0 0x7f08081986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41378/0 (socket says 172.21.15.132:41378) 2024-04-23T18:37:35.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080659c640 1 -- 172.21.15.132:0/1212130417 learned_addr learned my addr 172.21.15.132:0/1212130417 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:35.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f0805d9b640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808103af0 0x7f0808198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f0806d9d640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f080810c510 0x7f080819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080659c640 1 -- 172.21.15.132:0/1212130417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808103af0 msgr2=0x7f0808198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080659c640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808103af0 0x7f0808198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080659c640 1 -- 172.21.15.132:0/1212130417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f080810c510 msgr2=0x7f080819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080659c640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f080810c510 0x7f080819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080659c640 1 -- 172.21.15.132:0/1212130417 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080819d6d0 con 0x7f08081031d0 2024-04-23T18:37:35.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f0806d9d640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f080810c510 0x7f080819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:37:35.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080659c640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08081031d0 0x7f08081986e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f07f800e9f0 tx=0x7f07f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:35.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f07f77fe640 1 -- 172.21.15.132:0/1212130417 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f800cd80 con 0x7f08081031d0 2024-04-23T18:37:35.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f07f77fe640 1 -- 172.21.15.132:0/1212130417 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f80108e0 con 0x7f08081031d0 2024-04-23T18:37:35.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f080cbcd640 1 -- 172.21.15.132:0/1212130417 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080819d9c0 con 0x7f08081031d0 2024-04-23T18:37:35.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.488+0000 7f07f77fe640 1 -- 172.21.15.132:0/1212130417 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8018980 con 0x7f08081031d0 2024-04-23T18:37:35.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.489+0000 7f080cbcd640 1 -- 172.21.15.132:0/1212130417 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08081a5250 con 0x7f08081031d0 2024-04-23T18:37:35.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.490+0000 7f07f77fe640 1 -- 172.21.15.132:0/1212130417 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f07f8010430 con 0x7f08081031d0 2024-04-23T18:37:35.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.491+0000 7f080f9c3740 1 Processor -- start 2024-04-23T18:37:35.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.491+0000 7f07f77fe640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f07f00a3d20 0x7f07f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.491+0000 7f0805d9b640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f07f00a3d20 0x7f07f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.491+0000 7f07f77fe640 1 -- 172.21.15.132:0/1212130417 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f07f80d0b80 con 0x7f08081031d0 2024-04-23T18:37:35.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.491+0000 7f080f9c3740 1 -- start start 2024-04-23T18:37:35.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.491+0000 7f080f9c3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3cb360 0x555abc3cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f080f9c3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cbd10 0x555abc3d5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f080f9c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3d6380 0x555abc3d87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f080f9c3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x555abc331130 con 0x555abc3cbd10 2024-04-23T18:37:35.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f080f9c3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555abc2cdc50 con 0x555abc3cb360 2024-04-23T18:37:35.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f080f9c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555abc32d0f0 con 0x555abc3d6380 2024-04-23T18:37:35.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f0805d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cbd10 0x555abc3d5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f0805d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cbd10 0x555abc3d5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41388/0 (socket says 172.21.15.132:41388) 2024-04-23T18:37:35.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f0805d9b640 1 -- 172.21.15.132:0/302682559 learned_addr learned my addr 172.21.15.132:0/302682559 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:35.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f080659c640 1 --2- 172.21.15.132:0/302682559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3cb360 0x555abc3cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f0805d9b640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f07f00a3d20 0x7f07f00a61e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f080810e340 tx=0x7f07fc005f90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:35.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f0806d9d640 1 --2- 172.21.15.132:0/302682559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3d6380 0x555abc3d87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f0805d9b640 1 -- 172.21.15.132:0/302682559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3d6380 msgr2=0x555abc3d87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:35.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f0805d9b640 1 --2- 172.21.15.132:0/302682559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3d6380 0x555abc3d87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f0805d9b640 1 -- 172.21.15.132:0/302682559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3cb360 msgr2=0x555abc3cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:35.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f0805d9b640 1 --2- 172.21.15.132:0/302682559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3cb360 0x555abc3cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f0805d9b640 1 -- 172.21.15.132:0/302682559 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555abc2e7fd0 con 0x555abc3cbd10 2024-04-23T18:37:35.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f0805d9b640 1 --2- 172.21.15.132:0/302682559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cbd10 0x555abc3d5db0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f07fc011e40 tx=0x7f07fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378c0c0cb8a7e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:35.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f07f4ff9640 1 -- 172.21.15.132:0/302682559 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00ecb0 con 0x555abc3cbd10 2024-04-23T18:37:35.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f07f4ff9640 1 -- 172.21.15.132:0/302682559 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07fc00ee50 con 0x555abc3cbd10 2024-04-23T18:37:35.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.492+0000 7f07f4ff9640 1 -- 172.21.15.132:0/302682559 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc026480 con 0x555abc3cbd10 2024-04-23T18:37:35.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.493+0000 7f080f9c3740 1 -- 172.21.15.132:0/302682559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cbd10 msgr2=0x555abc3d5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.493+0000 7f080f9c3740 1 --2- 172.21.15.132:0/302682559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cbd10 0x555abc3d5db0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f07fc011e40 tx=0x7f07fc014000 comp rx=0 tx=0).stop 2024-04-23T18:37:35.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.493+0000 7f080f9c3740 1 -- 172.21.15.132:0/302682559 shutdown_connections 2024-04-23T18:37:35.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.493+0000 7f080f9c3740 1 --2- 172.21.15.132:0/302682559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3d6380 0x555abc3d87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.493+0000 7f080f9c3740 1 --2- 172.21.15.132:0/302682559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cbd10 0x555abc3d5db0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.493+0000 7f080f9c3740 1 --2- 172.21.15.132:0/302682559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3cb360 0x555abc3cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.493+0000 7f080f9c3740 1 -- 172.21.15.132:0/302682559 >> 172.21.15.132:0/302682559 conn(0x555abc3c7620 msgr2=0x555abc3c7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:35.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.493+0000 7f080f9c3740 1 -- 172.21.15.132:0/302682559 shutdown_connections 2024-04-23T18:37:35.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.493+0000 7f080f9c3740 1 -- 172.21.15.132:0/302682559 wait complete. 2024-04-23T18:37:35.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.493+0000 7f080f9c3740 1 -- start start 2024-04-23T18:37:35.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.493+0000 7f080f9c3740 1 Processor -- start 2024-04-23T18:37:35.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080f9c3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc3d32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080f9c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3cbd10 0x555abc3d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080f9c3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3d6380 0x555abc3d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080f9c3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x555abc3d92f0 con 0x555abc3cb360 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080f9c3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555abc2cb640 con 0x555abc3d6380 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080f9c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555abc3d9470 con 0x555abc3cbd10 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080f9c3740 10 client.0 _ll_register_callbacks cb 0x555abc2cb030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc3d32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc3d32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41394/0 (socket says 172.21.15.132:41394) 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080659c640 1 -- 172.21.15.132:0/1488513199 learned_addr learned my addr 172.21.15.132:0/1488513199 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f0806d9d640 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3d6380 0x555abc3d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f0805d9b640 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3cbd10 0x555abc3d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080659c640 1 -- 172.21.15.132:0/1488513199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3cbd10 msgr2=0x555abc3d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080659c640 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3cbd10 0x555abc3d37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080659c640 1 -- 172.21.15.132:0/1488513199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3d6380 msgr2=0x555abc3d3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080659c640 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3d6380 0x555abc3d3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080659c640 1 -- 172.21.15.132:0/1488513199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555abc10fa20 con 0x555abc3cb360 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f0805d9b640 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3cbd10 0x555abc3d37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080659c640 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc3d32b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f07f80994e0 tx=0x7f07f80107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cb5d8633a64a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f07edffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f07edffb640 1 -- 172.21.15.132:0/1488513199 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f80a0ec0 con 0x555abc3cb360 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f07edffb640 1 -- 172.21.15.132:0/1488513199 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f07f8099180 con 0x555abc3cb360 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080f9c3740 1 -- 172.21.15.132:0/1488513199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07fc00ee50 con 0x555abc3cb360 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f07edffb640 1 -- 172.21.15.132:0/1488513199 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f07f80a5780 con 0x555abc3cb360 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.494+0000 7f080f9c3740 10 client.15900 fetch_fsmap learned FSMap version 27 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.495+0000 7f080f9c3740 1 -- 172.21.15.132:0/1488513199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555abc1701b0 con 0x555abc3cb360 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.495+0000 7f07edffb640 1 -- 172.21.15.132:0/1488513199 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07fc00ee50 con 0x555abc3cb360 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.495+0000 7f07edffb640 1 -- 172.21.15.132:0/1488513199 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f07f80b8070 con 0x555abc3cb360 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.495+0000 7f080f9c3740 10 client.15900 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:37:35.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.495+0000 7f080f9c3740 10 client.15900 resolve_mds: validated gid 24515 aka daemon mds.b 2024-04-23T18:37:35.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.495+0000 7f080f9c3740 20 client.15900 populate_metadata read hostname 'smithi132' 2024-04-23T18:37:35.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.495+0000 7f080f9c3740 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555abc3e6960 0x555abc3e8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.495+0000 7f080f9c3740 4 client.15900 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:37:35.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.495+0000 7f080f9c3740 1 -- 172.21.15.132:0/1488513199 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555abc33eb50 con 0x555abc3e6960 2024-04-23T18:37:35.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.495+0000 7f0805d9b640 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555abc3e6960 0x555abc3e8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.495+0000 7f07edffb640 10 client.15900 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:35.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.495+0000 7f0805d9b640 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555abc3e6960 0x555abc3e8dd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f739b79fffc2a3e server_cookie=b7bce963c80c6c91 in_seq=0 out_seq=0 2024-04-23T18:37:35.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f07edffb640 1 -- 172.21.15.132:0/1488513199 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555abc33eb50 con 0x555abc3e6960 2024-04-23T18:37:35.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f07edffb640 10 client.15900 handle_command_reply: tid=0 2024-04-23T18:37:35.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f080f9c3740 1 client.15900 shutdown 2024-04-23T18:37:35.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f080f9c3740 1 -- 172.21.15.132:0/1488513199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 msgr2=0x555abc3d32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f080f9c3740 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc3d32b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f07f80994e0 tx=0x7f07f80107e0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f080f9c3740 1 -- 172.21.15.132:0/1488513199 shutdown_connections 2024-04-23T18:37:35.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f080f9c3740 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555abc3e6960 0x555abc3e8dd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f080f9c3740 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3cbd10 0x555abc3d37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f080f9c3740 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3d6380 0x555abc3d3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f080f9c3740 1 --2- 172.21.15.132:0/1488513199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc3d32b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f07edffb640 0 client.15900 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:35.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f080f9c3740 1 -- 172.21.15.132:0/1488513199 >> 172.21.15.132:0/1488513199 conn(0x555abc3c7620 msgr2=0x555abc3cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:35.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f080f9c3740 1 -- 172.21.15.132:0/1488513199 shutdown_connections 2024-04-23T18:37:35.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f080f9c3740 1 -- 172.21.15.132:0/1488513199 wait complete. 2024-04-23T18:37:35.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.496+0000 7f080f9c3740 20 client.15900 trim_cache size 0 max 16384 2024-04-23T18:37:35.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.513+0000 7f080f9c3740 1 Processor -- start 2024-04-23T18:37:35.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.513+0000 7f080f9c3740 1 -- start start 2024-04-23T18:37:35.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.513+0000 7f080f9c3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc3e2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.513+0000 7f080f9c3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3cbd10 0x555abc3deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.513+0000 7f080f9c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3d6380 0x555abc3ee650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.513+0000 7f080f9c3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f08081adc70 con 0x555abc3cb360 2024-04-23T18:37:35.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.513+0000 7f080f9c3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f08081af7e0 con 0x555abc3cbd10 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.513+0000 7f080f9c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08081ae340 con 0x555abc3d6380 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f080659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc3e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f080659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc3e2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41410/0 (socket says 172.21.15.132:41410) 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f080659c640 1 -- 172.21.15.132:0/4061475546 learned_addr learned my addr 172.21.15.132:0/4061475546 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f0806d9d640 1 --2- 172.21.15.132:0/4061475546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3d6380 0x555abc3ee650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f080659c640 1 -- 172.21.15.132:0/4061475546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3d6380 msgr2=0x555abc3ee650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f080659c640 1 --2- 172.21.15.132:0/4061475546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3d6380 0x555abc3ee650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f080659c640 1 -- 172.21.15.132:0/4061475546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3cbd10 msgr2=0x555abc3deb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f080659c640 1 --2- 172.21.15.132:0/4061475546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3cbd10 0x555abc3deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f080659c640 1 -- 172.21.15.132:0/4061475546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555abc003310 con 0x555abc3cb360 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f0806d9d640 1 --2- 172.21.15.132:0/4061475546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3d6380 0x555abc3ee650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f080659c640 1 --2- 172.21.15.132:0/4061475546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc3e2210 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f07f800e9c0 tx=0x7f07f80bd2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2981875747df6d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:35.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f07effff640 1 -- 172.21.15.132:0/4061475546 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8010a90 con 0x555abc3cb360 2024-04-23T18:37:35.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f07effff640 1 -- 172.21.15.132:0/4061475546 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f8099180 con 0x555abc3cb360 2024-04-23T18:37:35.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.514+0000 7f07effff640 1 -- 172.21.15.132:0/4061475546 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8018d80 con 0x555abc3cb360 2024-04-23T18:37:35.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 -- 172.21.15.132:0/4061475546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 msgr2=0x555abc3e2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 --2- 172.21.15.132:0/4061475546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc3e2210 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f07f800e9c0 tx=0x7f07f80bd2c0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 -- 172.21.15.132:0/4061475546 shutdown_connections 2024-04-23T18:37:35.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 --2- 172.21.15.132:0/4061475546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3d6380 0x555abc3ee650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 --2- 172.21.15.132:0/4061475546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3cbd10 0x555abc3deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 --2- 172.21.15.132:0/4061475546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc3e2210 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 -- 172.21.15.132:0/4061475546 >> 172.21.15.132:0/4061475546 conn(0x555abc3c7620 msgr2=0x555abc3c60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:35.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 -- 172.21.15.132:0/4061475546 shutdown_connections 2024-04-23T18:37:35.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 -- 172.21.15.132:0/4061475546 wait complete. 2024-04-23T18:37:35.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 -- start start 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 Processor -- start 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc445070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3cbd10 0x555abc4455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3d6380 0x555abc4495f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f08081af0e0 con 0x555abc3cb360 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f08081aea10 con 0x555abc3d6380 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08081abc50 con 0x555abc3cbd10 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080f9c3740 10 client.0 _ll_register_callbacks cb 0x7f07f809c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc445070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc445070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41416/0 (socket says 172.21.15.132:41416) 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f080659c640 1 -- 172.21.15.132:0/2600430795 learned_addr learned my addr 172.21.15.132:0/2600430795 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f0805d9b640 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3cbd10 0x555abc4455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.515+0000 7f0806d9d640 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3d6380 0x555abc4495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080659c640 1 -- 172.21.15.132:0/2600430795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3cbd10 msgr2=0x555abc4455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080659c640 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3cbd10 0x555abc4455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080659c640 1 -- 172.21.15.132:0/2600430795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3d6380 msgr2=0x555abc4495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080659c640 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3d6380 0x555abc4495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080659c640 1 -- 172.21.15.132:0/2600430795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555abc449c60 con 0x555abc3cb360 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f0806d9d640 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3d6380 0x555abc4495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080659c640 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc445070 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f07f8098b30 tx=0x7f07f8018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab1d084f51e28bb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f07edffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f07edffb640 1 -- 172.21.15.132:0/2600430795 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f809c160 con 0x555abc3cb360 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f07edffb640 1 -- 172.21.15.132:0/2600430795 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f07f8018b20 con 0x555abc3cb360 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080f9c3740 1 -- 172.21.15.132:0/2600430795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07f8099180 con 0x555abc3cb360 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f07edffb640 1 -- 172.21.15.132:0/2600430795 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f07f8018d20 con 0x555abc3cb360 2024-04-23T18:37:35.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080f9c3740 10 client.15906 fetch_fsmap learned FSMap version 27 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080f9c3740 1 -- 172.21.15.132:0/2600430795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555abc452150 con 0x555abc3cb360 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f07edffb640 1 -- 172.21.15.132:0/2600430795 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f8099180 con 0x555abc3cb360 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f07edffb640 1 -- 172.21.15.132:0/2600430795 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f07f80b8070 con 0x555abc3cb360 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080f9c3740 10 client.15906 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080f9c3740 10 client.15906 resolve_mds: validated gid 24515 aka daemon mds.b 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080f9c3740 20 client.15906 populate_metadata read hostname 'smithi132' 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080f9c3740 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555abc4526a0 0x555abc454ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080f9c3740 4 client.15906 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.516+0000 7f080f9c3740 1 -- 172.21.15.132:0/2600430795 --> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x555abc3389c0 con 0x555abc4526a0 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f0805d9b640 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555abc4526a0 0x555abc454ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f07edffb640 10 client.15906 ms_handle_connect on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f0805d9b640 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555abc4526a0 0x555abc454ac0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850fe14216de371c server_cookie=c7939885b7b9c44b in_seq=0 out_seq=0 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f07edffb640 1 -- 172.21.15.132:0/2600430795 <== mds.0 v2:172.21.15.148:6838/4115412726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x555abc3389c0 con 0x555abc4526a0 2024-04-23T18:37:35.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f07edffb640 10 client.15906 handle_command_reply: tid=0 2024-04-23T18:37:35.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f080f9c3740 1 client.15906 shutdown 2024-04-23T18:37:35.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f080f9c3740 1 -- 172.21.15.132:0/2600430795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 msgr2=0x555abc445070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f080f9c3740 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc445070 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f07f8098b30 tx=0x7f07f8018420 comp rx=0 tx=0).stop 2024-04-23T18:37:35.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f080f9c3740 1 -- 172.21.15.132:0/2600430795 shutdown_connections 2024-04-23T18:37:35.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f080f9c3740 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.148:6838/4115412726,v1:172.21.15.148:6839/4115412726] conn(0x555abc4526a0 0x555abc454ac0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f080f9c3740 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555abc3cbd10 0x555abc4455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f07edffb640 0 client.15906 ms_handle_reset on v2:172.21.15.148:6838/4115412726 2024-04-23T18:37:35.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f080f9c3740 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555abc3d6380 0x555abc4495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f080f9c3740 1 --2- 172.21.15.132:0/2600430795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555abc3cb360 0x555abc445070 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f080f9c3740 1 -- 172.21.15.132:0/2600430795 >> 172.21.15.132:0/2600430795 conn(0x555abc3c7620 msgr2=0x555abc3c6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:35.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f080f9c3740 1 -- 172.21.15.132:0/2600430795 shutdown_connections 2024-04-23T18:37:35.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f080f9c3740 1 -- 172.21.15.132:0/2600430795 wait complete. 2024-04-23T18:37:35.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.517+0000 7f080f9c3740 20 client.15906 trim_cache size 0 max 16384 2024-04-23T18:37:35.519 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:37:35.519 INFO:teuthology.orchestra.run.smithi132.stdout: "epoch": 27, 2024-04-23T18:37:35.519 INFO:teuthology.orchestra.run.smithi132.stdout: "leader": 24515, 2024-04-23T18:37:35.519 INFO:teuthology.orchestra.run.smithi132.stdout: "set_version": 8, 2024-04-23T18:37:35.519 INFO:teuthology.orchestra.run.smithi132.stdout: "sets": {} 2024-04-23T18:37:35.519 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:37:35.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.518+0000 7f080cbcd640 1 -- 172.21.15.132:0/1212130417 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f07f00a3d20 msgr2=0x7f07f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.518+0000 7f080cbcd640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f07f00a3d20 0x7f07f00a61e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f080810e340 tx=0x7f07fc005f90 comp rx=0 tx=0).stop 2024-04-23T18:37:35.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.518+0000 7f080cbcd640 1 -- 172.21.15.132:0/1212130417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08081031d0 msgr2=0x7f08081986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:35.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.518+0000 7f080cbcd640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08081031d0 0x7f08081986e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f07f800e9f0 tx=0x7f07f800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.518+0000 7f080cbcd640 1 -- 172.21.15.132:0/1212130417 shutdown_connections 2024-04-23T18:37:35.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.518+0000 7f080cbcd640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f07f00a3d20 0x7f07f00a61e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.518+0000 7f080cbcd640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808103af0 0x7f0808198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.518+0000 7f080cbcd640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f080810c510 0x7f080819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.518+0000 7f080cbcd640 1 --2- 172.21.15.132:0/1212130417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08081031d0 0x7f08081986e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.518+0000 7f080cbcd640 1 -- 172.21.15.132:0/1212130417 >> 172.21.15.132:0/1212130417 conn(0x7f0808077f50 msgr2=0x7f08080fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:35.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.518+0000 7f080cbcd640 1 -- 172.21.15.132:0/1212130417 shutdown_connections 2024-04-23T18:37:35.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:37:35.518+0000 7f080cbcd640 1 -- 172.21.15.132:0/1212130417 wait complete. 2024-04-23T18:37:35.522 INFO:tasks.quiescer:done joining Quiescers 2024-04-23T18:37:35.522 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-23T18:37:35.534 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi132.front.sepia.ceph.com 2024-04-23T18:37:35.542 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-23T18:37:35.552 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-23T18:37:35.553 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:37:35.553 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-04-23T18:37:35.553 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-23T18:37:35.608 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-23T18:37:36.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:36 smithi148 ceph-mon[32311]: pgmap v49: 97 pgs: 97 active+clean; 386 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 186 MiB/s rd, 2.0 MiB/s wr, 710 op/s 2024-04-23T18:37:36.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:36 smithi148 ceph-mon[32311]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-23T18:37:36.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:36 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/306502811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:37:36.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:36 smithi157 ceph-mon[32580]: pgmap v49: 97 pgs: 97 active+clean; 386 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 186 MiB/s rd, 2.0 MiB/s wr, 710 op/s 2024-04-23T18:37:36.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:36 smithi157 ceph-mon[32580]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-23T18:37:36.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:36 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/306502811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:37:36.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:36 smithi132 ceph-mon[29178]: pgmap v49: 97 pgs: 97 active+clean; 386 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 186 MiB/s rd, 2.0 MiB/s wr, 710 op/s 2024-04-23T18:37:36.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:36 smithi132 ceph-mon[29178]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-23T18:37:36.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:36 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/306502811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:37:37.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:37 smithi148 ceph-mon[32311]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:37:37.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:37 smithi157 ceph-mon[32580]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:37:37.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:37 smithi132 ceph-mon[29178]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:37:38.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:38 smithi148 ceph-mon[32311]: pgmap v50: 97 pgs: 97 active+clean; 386 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 137 MiB/s rd, 1.0 MiB/s wr, 422 op/s 2024-04-23T18:37:38.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:38 smithi157 ceph-mon[32580]: pgmap v50: 97 pgs: 97 active+clean; 386 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 137 MiB/s rd, 1.0 MiB/s wr, 422 op/s 2024-04-23T18:37:38.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:38 smithi132 ceph-mon[29178]: pgmap v50: 97 pgs: 97 active+clean; 386 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 137 MiB/s rd, 1.0 MiB/s wr, 422 op/s 2024-04-23T18:37:40.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:40 smithi148 ceph-mon[32311]: pgmap v51: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 139 MiB/s rd, 1.1 MiB/s wr, 488 op/s 2024-04-23T18:37:40.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:40 smithi157 ceph-mon[32580]: pgmap v51: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 139 MiB/s rd, 1.1 MiB/s wr, 488 op/s 2024-04-23T18:37:40.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:40 smithi132 ceph-mon[29178]: pgmap v51: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 139 MiB/s rd, 1.1 MiB/s wr, 488 op/s 2024-04-23T18:37:42.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:42 smithi148 ceph-mon[32311]: pgmap v52: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 636 KiB/s wr, 275 op/s 2024-04-23T18:37:42.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:42 smithi157 ceph-mon[32580]: pgmap v52: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 636 KiB/s wr, 275 op/s 2024-04-23T18:37:42.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:42 smithi132 ceph-mon[29178]: pgmap v52: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 636 KiB/s wr, 275 op/s 2024-04-23T18:37:44.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:44 smithi148 ceph-mon[32311]: pgmap v53: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 636 KiB/s wr, 275 op/s 2024-04-23T18:37:44.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:44 smithi157 ceph-mon[32580]: pgmap v53: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 636 KiB/s wr, 275 op/s 2024-04-23T18:37:44.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:44 smithi132 ceph-mon[29178]: pgmap v53: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 636 KiB/s wr, 275 op/s 2024-04-23T18:37:46.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:46 smithi148 ceph-mon[32311]: pgmap v54: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 636 KiB/s wr, 280 op/s 2024-04-23T18:37:46.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:46 smithi157 ceph-mon[32580]: pgmap v54: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 636 KiB/s wr, 280 op/s 2024-04-23T18:37:46.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:46 smithi132 ceph-mon[29178]: pgmap v54: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 636 KiB/s wr, 280 op/s 2024-04-23T18:37:48.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:48 smithi148 ceph-mon[32311]: pgmap v55: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 123 KiB/s wr, 71 op/s 2024-04-23T18:37:48.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:48 smithi157 ceph-mon[32580]: pgmap v55: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 123 KiB/s wr, 71 op/s 2024-04-23T18:37:48.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:48 smithi132 ceph-mon[29178]: pgmap v55: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 123 KiB/s wr, 71 op/s 2024-04-23T18:37:50.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:50 smithi148 ceph-mon[32311]: pgmap v56: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 124 KiB/s wr, 75 op/s 2024-04-23T18:37:50.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:50 smithi157 ceph-mon[32580]: pgmap v56: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 124 KiB/s wr, 75 op/s 2024-04-23T18:37:50.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:50 smithi132 ceph-mon[29178]: pgmap v56: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 124 KiB/s wr, 75 op/s 2024-04-23T18:37:52.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:52 smithi148 ceph-mon[32311]: pgmap v57: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-23T18:37:52.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:52 smithi157 ceph-mon[32580]: pgmap v57: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-23T18:37:52.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:52 smithi132 ceph-mon[29178]: pgmap v57: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-23T18:37:54.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:54 smithi148 ceph-mon[32311]: pgmap v58: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-23T18:37:54.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:54 smithi157 ceph-mon[32580]: pgmap v58: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-23T18:37:54.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:54 smithi132 ceph-mon[29178]: pgmap v58: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-23T18:37:56.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:56 smithi148 ceph-mon[32311]: pgmap v59: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-04-23T18:37:56.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:56 smithi157 ceph-mon[32580]: pgmap v59: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-04-23T18:37:56.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:56 smithi132 ceph-mon[29178]: pgmap v59: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-04-23T18:37:58.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:37:58 smithi148 ceph-mon[32311]: pgmap v60: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-23T18:37:58.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:37:58 smithi157 ceph-mon[32580]: pgmap v60: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-23T18:37:58.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:37:58 smithi132 ceph-mon[29178]: pgmap v60: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-23T18:38:00.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:00 smithi148 ceph-mon[32311]: pgmap v61: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 14 op/s 2024-04-23T18:38:00.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:00 smithi157 ceph-mon[32580]: pgmap v61: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 14 op/s 2024-04-23T18:38:00.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:00 smithi132 ceph-mon[29178]: pgmap v61: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 14 op/s 2024-04-23T18:38:02.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:02 smithi148 ceph-mon[32311]: pgmap v62: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:02.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:02 smithi157 ceph-mon[32580]: pgmap v62: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:02.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:02 smithi132 ceph-mon[29178]: pgmap v62: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:04.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:04 smithi148 ceph-mon[32311]: pgmap v63: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:04.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:04 smithi157 ceph-mon[32580]: pgmap v63: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:04.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:04 smithi132 ceph-mon[29178]: pgmap v63: 97 pgs: 97 active+clean; 9.8 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:06.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:06 smithi132 ceph-mon[29178]: pgmap v64: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:38:07.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:06 smithi148 ceph-mon[32311]: pgmap v64: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:38:07.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:06 smithi157 ceph-mon[32580]: pgmap v64: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:38:08.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:08 smithi132 ceph-mon[29178]: pgmap v65: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:38:09.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:08 smithi148 ceph-mon[32311]: pgmap v65: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:38:09.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:08 smithi157 ceph-mon[32580]: pgmap v65: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:38:09.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:09 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:38:09.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:09 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:09.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:09 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:09.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:09 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:09.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:09 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:10.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:09 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:38:10.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:09 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:10.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:09 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:10.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:09 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:10.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:09 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:10.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:09 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:38:10.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:09 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:10.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:09 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:10.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:09 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:10.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:09 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:10.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:10 smithi132 ceph-mon[29178]: pgmap v66: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:38:10.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:10 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:38:10.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:10 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:38:10.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:10 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:10.915 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:10 smithi148 ceph-mon[32311]: pgmap v66: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:38:10.915 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:10 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:38:10.915 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:10 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:38:10.915 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:10 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:11.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:10 smithi157 ceph-mon[32580]: pgmap v66: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:38:11.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:10 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:38:11.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:10 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:38:11.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:10 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:38:12.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:12 smithi132 ceph-mon[29178]: pgmap v67: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:38:13.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:12 smithi148 ceph-mon[32311]: pgmap v67: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:38:13.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:12 smithi157 ceph-mon[32580]: pgmap v67: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:38:14.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:14 smithi132 ceph-mon[29178]: pgmap v68: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:38:15.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:14 smithi148 ceph-mon[32311]: pgmap v68: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:38:15.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:14 smithi157 ceph-mon[32580]: pgmap v68: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:38:16.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:16 smithi132 ceph-mon[29178]: pgmap v69: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:38:17.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:16 smithi148 ceph-mon[32311]: pgmap v69: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:38:17.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:16 smithi157 ceph-mon[32580]: pgmap v69: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:38:18.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:18 smithi132 ceph-mon[29178]: pgmap v70: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:38:19.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:18 smithi148 ceph-mon[32311]: pgmap v70: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:38:19.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:18 smithi157 ceph-mon[32580]: pgmap v70: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:38:20.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:20 smithi132 ceph-mon[29178]: pgmap v71: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-23T18:38:21.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:20 smithi148 ceph-mon[32311]: pgmap v71: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-23T18:38:21.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:20 smithi157 ceph-mon[32580]: pgmap v71: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-23T18:38:22.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:22 smithi132 ceph-mon[29178]: pgmap v72: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:38:23.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:22 smithi148 ceph-mon[32311]: pgmap v72: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:38:23.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:22 smithi157 ceph-mon[32580]: pgmap v72: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:38:24.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:24 smithi132 ceph-mon[29178]: pgmap v73: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:38:25.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:24 smithi148 ceph-mon[32311]: pgmap v73: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:38:25.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:24 smithi157 ceph-mon[32580]: pgmap v73: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:38:26.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:26 smithi132 ceph-mon[29178]: pgmap v74: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:38:27.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:26 smithi148 ceph-mon[32311]: pgmap v74: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:38:27.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:26 smithi157 ceph-mon[32580]: pgmap v74: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:38:28.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:28 smithi132 ceph-mon[29178]: pgmap v75: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:29.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:28 smithi148 ceph-mon[32311]: pgmap v75: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:29.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:28 smithi157 ceph-mon[32580]: pgmap v75: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:30.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:30 smithi132 ceph-mon[29178]: pgmap v76: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:31.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:30 smithi148 ceph-mon[32311]: pgmap v76: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:31.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:30 smithi157 ceph-mon[32580]: pgmap v76: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:32.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:32 smithi132 ceph-mon[29178]: pgmap v77: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:33.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:32 smithi148 ceph-mon[32311]: pgmap v77: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:33.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:32 smithi157 ceph-mon[32580]: pgmap v77: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:34.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:34 smithi132 ceph-mon[29178]: pgmap v78: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:35.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:34 smithi148 ceph-mon[32311]: pgmap v78: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:35.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:34 smithi157 ceph-mon[32580]: pgmap v78: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:36.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:36 smithi132 ceph-mon[29178]: pgmap v79: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:37.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:36 smithi148 ceph-mon[32311]: pgmap v79: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:37.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:36 smithi157 ceph-mon[32580]: pgmap v79: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:39.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:38 smithi148 ceph-mon[32311]: pgmap v80: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:39.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:38 smithi157 ceph-mon[32580]: pgmap v80: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:39.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:38 smithi132 ceph-mon[29178]: pgmap v80: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:41.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:40 smithi148 ceph-mon[32311]: pgmap v81: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:41.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:40 smithi157 ceph-mon[32580]: pgmap v81: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:41.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:40 smithi132 ceph-mon[29178]: pgmap v81: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:43.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:42 smithi148 ceph-mon[32311]: pgmap v82: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:43.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:42 smithi157 ceph-mon[32580]: pgmap v82: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:43.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:42 smithi132 ceph-mon[29178]: pgmap v82: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:45.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:44 smithi148 ceph-mon[32311]: pgmap v83: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:45.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:44 smithi157 ceph-mon[32580]: pgmap v83: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:45.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:44 smithi132 ceph-mon[29178]: pgmap v83: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:47.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:46 smithi148 ceph-mon[32311]: pgmap v84: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:47.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:46 smithi157 ceph-mon[32580]: pgmap v84: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:47.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:46 smithi132 ceph-mon[29178]: pgmap v84: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:48 smithi148 ceph-mon[32311]: pgmap v85: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:48 smithi157 ceph-mon[32580]: pgmap v85: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:48 smithi132 ceph-mon[29178]: pgmap v85: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:51.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:50 smithi148 ceph-mon[32311]: pgmap v86: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:51.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:50 smithi157 ceph-mon[32580]: pgmap v86: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:51.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:50 smithi132 ceph-mon[29178]: pgmap v86: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:53.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:52 smithi148 ceph-mon[32311]: pgmap v87: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:53.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:52 smithi157 ceph-mon[32580]: pgmap v87: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:53.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:52 smithi132 ceph-mon[29178]: pgmap v87: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:55.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:54 smithi148 ceph-mon[32311]: pgmap v88: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:55.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:54 smithi157 ceph-mon[32580]: pgmap v88: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:55.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:54 smithi132 ceph-mon[29178]: pgmap v88: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:56.913 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:56 smithi132 ceph-mon[29178]: pgmap v89: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:57.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:56 smithi148 ceph-mon[32311]: pgmap v89: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:57.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:56 smithi157 ceph-mon[32580]: pgmap v89: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:38:59.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:38:58 smithi148 ceph-mon[32311]: pgmap v90: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:59.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:38:58 smithi157 ceph-mon[32580]: pgmap v90: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:38:59.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:38:58 smithi132 ceph-mon[29178]: pgmap v90: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:01.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:00 smithi148 ceph-mon[32311]: pgmap v91: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:01.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:00 smithi157 ceph-mon[32580]: pgmap v91: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:01.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:00 smithi132 ceph-mon[29178]: pgmap v91: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:03.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:02 smithi148 ceph-mon[32311]: pgmap v92: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:03.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:02 smithi157 ceph-mon[32580]: pgmap v92: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:03.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:02 smithi132 ceph-mon[29178]: pgmap v92: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:04.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:03 smithi148 ceph-mon[32311]: pgmap v93: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:04.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:03 smithi157 ceph-mon[32580]: pgmap v93: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:04.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:03 smithi132 ceph-mon[29178]: pgmap v93: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:06.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:05 smithi132 ceph-mon[29178]: pgmap v94: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:39:06.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:05 smithi148 ceph-mon[32311]: pgmap v94: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:39:06.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:05 smithi157 ceph-mon[32580]: pgmap v94: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:39:08.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:07 smithi132 ceph-mon[29178]: pgmap v95: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:39:08.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:07 smithi148 ceph-mon[32311]: pgmap v95: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:39:08.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:07 smithi157 ceph-mon[32580]: pgmap v95: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:39:10.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:09 smithi132 ceph-mon[29178]: pgmap v96: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:39:10.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:09 smithi148 ceph-mon[32311]: pgmap v96: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:39:10.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:09 smithi157 ceph-mon[32580]: pgmap v96: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:39:11.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:10 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:39:11.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:10 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:39:11.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:10 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:39:12.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:11 smithi132 ceph-mon[29178]: pgmap v97: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:39:12.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:11 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:39:12.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:11 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:39:12.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:11 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:39:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:11 smithi148 ceph-mon[32311]: pgmap v97: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:39:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:11 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:39:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:11 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:39:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:11 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:39:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:11 smithi157 ceph-mon[32580]: pgmap v97: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:39:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:11 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:39:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:11 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:39:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:11 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:39:14.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:13 smithi132 ceph-mon[29178]: pgmap v98: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:39:14.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:13 smithi148 ceph-mon[32311]: pgmap v98: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:39:14.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:13 smithi157 ceph-mon[32580]: pgmap v98: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:39:16.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:15 smithi132 ceph-mon[29178]: pgmap v99: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:39:16.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:15 smithi148 ceph-mon[32311]: pgmap v99: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:39:16.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:15 smithi157 ceph-mon[32580]: pgmap v99: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:39:18.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:17 smithi132 ceph-mon[29178]: pgmap v100: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:39:18.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:17 smithi148 ceph-mon[32311]: pgmap v100: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:39:18.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:17 smithi157 ceph-mon[32580]: pgmap v100: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:39:20.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:19 smithi132 ceph-mon[29178]: pgmap v101: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-23T18:39:20.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:19 smithi148 ceph-mon[32311]: pgmap v101: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-23T18:39:20.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:19 smithi157 ceph-mon[32580]: pgmap v101: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-23T18:39:22.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:21 smithi132 ceph-mon[29178]: pgmap v102: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:39:22.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:21 smithi148 ceph-mon[32311]: pgmap v102: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:39:22.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:21 smithi157 ceph-mon[32580]: pgmap v102: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:39:24.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:23 smithi132 ceph-mon[29178]: pgmap v103: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:39:24.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:23 smithi148 ceph-mon[32311]: pgmap v103: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:39:24.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:23 smithi157 ceph-mon[32580]: pgmap v103: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:39:26.267 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:25 smithi148 ceph-mon[32311]: pgmap v104: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:39:26.267 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:25 smithi148 ceph-mon[32311]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:39:26.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:25 smithi157 ceph-mon[32580]: pgmap v104: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:39:26.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:25 smithi157 ceph-mon[32580]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:39:26.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:25 smithi132 ceph-mon[29178]: pgmap v104: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:39:26.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:25 smithi132 ceph-mon[29178]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:39:28.163 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:27 smithi132 ceph-mon[29178]: pgmap v105: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:28.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:27 smithi148 ceph-mon[32311]: pgmap v105: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:28.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:27 smithi157 ceph-mon[32580]: pgmap v105: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:30.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:29 smithi148 ceph-mon[32311]: pgmap v106: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:30.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:29 smithi157 ceph-mon[32580]: pgmap v106: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:30.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:29 smithi132 ceph-mon[29178]: pgmap v106: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:32.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:31 smithi148 ceph-mon[32311]: pgmap v107: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:32.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:31 smithi157 ceph-mon[32580]: pgmap v107: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:32.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:31 smithi132 ceph-mon[29178]: pgmap v107: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:34.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:33 smithi148 ceph-mon[32311]: pgmap v108: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:34.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:33 smithi157 ceph-mon[32580]: pgmap v108: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:34.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:33 smithi132 ceph-mon[29178]: pgmap v108: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:36.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:35 smithi148 ceph-mon[32311]: pgmap v109: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:36.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:35 smithi157 ceph-mon[32580]: pgmap v109: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:36.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:35 smithi132 ceph-mon[29178]: pgmap v109: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:37.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:36 smithi148 ceph-mon[32311]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:39:37.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:36 smithi157 ceph-mon[32580]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:39:37.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:36 smithi132 ceph-mon[29178]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:39:38.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:37 smithi148 ceph-mon[32311]: pgmap v110: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:38.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:37 smithi157 ceph-mon[32580]: pgmap v110: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:38.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:37 smithi132 ceph-mon[29178]: pgmap v110: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:40.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:39 smithi148 ceph-mon[32311]: pgmap v111: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:40.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:39 smithi157 ceph-mon[32580]: pgmap v111: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:40.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:39 smithi132 ceph-mon[29178]: pgmap v111: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:42.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:41 smithi148 ceph-mon[32311]: pgmap v112: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:42.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:41 smithi157 ceph-mon[32580]: pgmap v112: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:42.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:41 smithi132 ceph-mon[29178]: pgmap v112: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:44.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:43 smithi148 ceph-mon[32311]: pgmap v113: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:44.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:43 smithi157 ceph-mon[32580]: pgmap v113: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:44.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:43 smithi132 ceph-mon[29178]: pgmap v113: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:46.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:45 smithi148 ceph-mon[32311]: pgmap v114: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:46.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:45 smithi157 ceph-mon[32580]: pgmap v114: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:46.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:45 smithi132 ceph-mon[29178]: pgmap v114: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:48.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:47 smithi148 ceph-mon[32311]: pgmap v115: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:48.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:47 smithi157 ceph-mon[32580]: pgmap v115: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:48.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:47 smithi132 ceph-mon[29178]: pgmap v115: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:50.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:49 smithi148 ceph-mon[32311]: pgmap v116: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:50.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:49 smithi157 ceph-mon[32580]: pgmap v116: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:50.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:49 smithi132 ceph-mon[29178]: pgmap v116: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:52.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:51 smithi148 ceph-mon[32311]: pgmap v117: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:52.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:51 smithi157 ceph-mon[32580]: pgmap v117: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:52.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:51 smithi132 ceph-mon[29178]: pgmap v117: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:54.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:53 smithi148 ceph-mon[32311]: pgmap v118: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:54.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:53 smithi157 ceph-mon[32580]: pgmap v118: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:54.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:53 smithi132 ceph-mon[29178]: pgmap v118: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:56.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:55 smithi148 ceph-mon[32311]: pgmap v119: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:56.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:55 smithi157 ceph-mon[32580]: pgmap v119: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:56.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:55 smithi132 ceph-mon[29178]: pgmap v119: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:39:58.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:57 smithi148 ceph-mon[32311]: pgmap v120: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:58.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:57 smithi157 ceph-mon[32580]: pgmap v120: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:39:58.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:57 smithi132 ceph-mon[29178]: pgmap v120: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:00.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:39:59 smithi148 ceph-mon[32311]: pgmap v121: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:00.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:39:59 smithi157 ceph-mon[32580]: pgmap v121: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:00.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:39:59 smithi132 ceph-mon[29178]: pgmap v121: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:01.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:00 smithi148 ceph-mon[32311]: overall HEALTH_OK 2024-04-23T18:40:01.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:00 smithi157 ceph-mon[32580]: overall HEALTH_OK 2024-04-23T18:40:01.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:00 smithi132 ceph-mon[29178]: overall HEALTH_OK 2024-04-23T18:40:02.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:01 smithi148 ceph-mon[32311]: pgmap v122: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:02.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:01 smithi157 ceph-mon[32580]: pgmap v122: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:02.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:01 smithi132 ceph-mon[29178]: pgmap v122: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:04.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:03 smithi148 ceph-mon[32311]: pgmap v123: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:04.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:03 smithi157 ceph-mon[32580]: pgmap v123: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:04.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:03 smithi132 ceph-mon[29178]: pgmap v123: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:06.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:05 smithi148 ceph-mon[32311]: pgmap v124: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:40:06.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:05 smithi157 ceph-mon[32580]: pgmap v124: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:40:06.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:05 smithi132 ceph-mon[29178]: pgmap v124: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:40:08.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:07 smithi148 ceph-mon[32311]: pgmap v125: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:40:08.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:07 smithi157 ceph-mon[32580]: pgmap v125: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:40:08.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:07 smithi132 ceph-mon[29178]: pgmap v125: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:40:10.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:09 smithi148 ceph-mon[32311]: pgmap v126: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:40:10.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:09 smithi157 ceph-mon[32580]: pgmap v126: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:40:10.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:09 smithi132 ceph-mon[29178]: pgmap v126: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:40:12.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:11 smithi148 ceph-mon[32311]: pgmap v127: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:40:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:11 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:40:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:11 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:40:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:11 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:40:12.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:11 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:40:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:11 smithi157 ceph-mon[32580]: pgmap v127: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:40:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:11 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:40:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:11 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:40:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:11 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:40:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:11 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:11 smithi132 ceph-mon[29178]: pgmap v127: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:11 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:11 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:11 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:11 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:40:14.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:13 smithi148 ceph-mon[32311]: pgmap v128: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:40:14.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:13 smithi157 ceph-mon[32580]: pgmap v128: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:40:14.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:13 smithi132 ceph-mon[29178]: pgmap v128: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:40:16.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:15 smithi148 ceph-mon[32311]: pgmap v129: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:40:16.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:15 smithi157 ceph-mon[32580]: pgmap v129: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:40:16.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:15 smithi132 ceph-mon[29178]: pgmap v129: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:40:18.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:17 smithi148 ceph-mon[32311]: pgmap v130: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:40:18.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:17 smithi157 ceph-mon[32580]: pgmap v130: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:40:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:17 smithi132 ceph-mon[29178]: pgmap v130: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:40:20.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:19 smithi148 ceph-mon[32311]: pgmap v131: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-23T18:40:20.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:19 smithi157 ceph-mon[32580]: pgmap v131: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-23T18:40:20.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:19 smithi132 ceph-mon[29178]: pgmap v131: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-23T18:40:22.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:21 smithi148 ceph-mon[32311]: pgmap v132: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:40:22.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:21 smithi157 ceph-mon[32580]: pgmap v132: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:40:22.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:21 smithi132 ceph-mon[29178]: pgmap v132: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:40:24.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:23 smithi148 ceph-mon[32311]: pgmap v133: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:40:24.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:23 smithi157 ceph-mon[32580]: pgmap v133: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:40:24.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:23 smithi132 ceph-mon[29178]: pgmap v133: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:40:26.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:25 smithi148 ceph-mon[32311]: pgmap v134: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:40:26.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:25 smithi157 ceph-mon[32580]: pgmap v134: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:40:26.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:25 smithi132 ceph-mon[29178]: pgmap v134: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:40:28.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:27 smithi148 ceph-mon[32311]: pgmap v135: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:28.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:27 smithi157 ceph-mon[32580]: pgmap v135: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:28.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:27 smithi132 ceph-mon[29178]: pgmap v135: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:30.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:29 smithi148 ceph-mon[32311]: pgmap v136: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:30.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:29 smithi157 ceph-mon[32580]: pgmap v136: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:30.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:29 smithi132 ceph-mon[29178]: pgmap v136: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:32.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:31 smithi148 ceph-mon[32311]: pgmap v137: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:32.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:31 smithi157 ceph-mon[32580]: pgmap v137: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:32.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:31 smithi132 ceph-mon[29178]: pgmap v137: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:34.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:33 smithi148 ceph-mon[32311]: pgmap v138: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:34.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:33 smithi157 ceph-mon[32580]: pgmap v138: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:34.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:33 smithi132 ceph-mon[29178]: pgmap v138: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:36.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:35 smithi148 ceph-mon[32311]: pgmap v139: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:36.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:35 smithi157 ceph-mon[32580]: pgmap v139: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:36.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:35 smithi132 ceph-mon[29178]: pgmap v139: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:38.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:37 smithi148 ceph-mon[32311]: pgmap v140: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:38.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:37 smithi157 ceph-mon[32580]: pgmap v140: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:38.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:37 smithi132 ceph-mon[29178]: pgmap v140: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:40.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:39 smithi148 ceph-mon[32311]: pgmap v141: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:40.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:39 smithi157 ceph-mon[32580]: pgmap v141: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:40.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:39 smithi132 ceph-mon[29178]: pgmap v141: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:42.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:42 smithi148 ceph-mon[32311]: pgmap v142: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:42.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:42 smithi157 ceph-mon[32580]: pgmap v142: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:42.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:42 smithi132 ceph-mon[29178]: pgmap v142: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:44.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:44 smithi148 ceph-mon[32311]: pgmap v143: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:44.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:44 smithi157 ceph-mon[32580]: pgmap v143: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:44.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:44 smithi132 ceph-mon[29178]: pgmap v143: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:46.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:46 smithi148 ceph-mon[32311]: pgmap v144: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:46.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:46 smithi157 ceph-mon[32580]: pgmap v144: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:46.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:46 smithi132 ceph-mon[29178]: pgmap v144: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:48.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:48 smithi148 ceph-mon[32311]: pgmap v145: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:48.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:48 smithi157 ceph-mon[32580]: pgmap v145: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:48.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:48 smithi132 ceph-mon[29178]: pgmap v145: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:50.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:50 smithi148 ceph-mon[32311]: pgmap v146: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:50.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:50 smithi157 ceph-mon[32580]: pgmap v146: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:50.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:50 smithi132 ceph-mon[29178]: pgmap v146: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:52.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:52 smithi148 ceph-mon[32311]: pgmap v147: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:52.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:52 smithi157 ceph-mon[32580]: pgmap v147: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:52.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:52 smithi132 ceph-mon[29178]: pgmap v147: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:54.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:54 smithi148 ceph-mon[32311]: pgmap v148: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:54.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:54 smithi157 ceph-mon[32580]: pgmap v148: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:54.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:54 smithi132 ceph-mon[29178]: pgmap v148: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:56.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:56 smithi157 ceph-mon[32580]: pgmap v149: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:56.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:56 smithi132 ceph-mon[29178]: pgmap v149: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:56.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:56 smithi148 ceph-mon[32311]: pgmap v149: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:40:58.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:40:58 smithi157 ceph-mon[32580]: pgmap v150: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:58.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:40:58 smithi132 ceph-mon[29178]: pgmap v150: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:40:58.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:40:58 smithi148 ceph-mon[32311]: pgmap v150: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:00.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:00 smithi157 ceph-mon[32580]: pgmap v151: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:00.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:00 smithi132 ceph-mon[29178]: pgmap v151: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:00.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:00 smithi148 ceph-mon[32311]: pgmap v151: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:02.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:02 smithi157 ceph-mon[32580]: pgmap v152: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:02.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:02 smithi132 ceph-mon[29178]: pgmap v152: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:02.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:02 smithi148 ceph-mon[32311]: pgmap v152: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:04.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:04 smithi157 ceph-mon[32580]: pgmap v153: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:04.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:04 smithi132 ceph-mon[29178]: pgmap v153: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:04.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:04 smithi148 ceph-mon[32311]: pgmap v153: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:06.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:06 smithi157 ceph-mon[32580]: pgmap v154: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:41:06.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:06 smithi132 ceph-mon[29178]: pgmap v154: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:41:06.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:06 smithi148 ceph-mon[32311]: pgmap v154: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:41:08.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:08 smithi157 ceph-mon[32580]: pgmap v155: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:41:08.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:08 smithi132 ceph-mon[29178]: pgmap v155: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:41:08.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:08 smithi148 ceph-mon[32311]: pgmap v155: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:41:10.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:10 smithi157 ceph-mon[32580]: pgmap v156: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:41:10.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:10 smithi148 ceph-mon[32311]: pgmap v156: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:41:10.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:10 smithi132 ceph-mon[29178]: pgmap v156: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:41:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:12 smithi157 ceph-mon[32580]: pgmap v157: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:41:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:12 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:41:12.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:12 smithi132 ceph-mon[29178]: pgmap v157: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:41:12.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:12 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:41:12.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:12 smithi148 ceph-mon[32311]: pgmap v157: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:41:12.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:12 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:41:13.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:13 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:41:13.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:13 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:41:13.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:13 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:41:13.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:13 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:41:13.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:13 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:41:13.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:13 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:41:13.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:13 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:41:13.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:13 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:41:13.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:13 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:41:14.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:14 smithi148 ceph-mon[32311]: pgmap v158: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:41:14.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:14 smithi157 ceph-mon[32580]: pgmap v158: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:41:14.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:14 smithi132 ceph-mon[29178]: pgmap v158: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:41:16.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:16 smithi148 ceph-mon[32311]: pgmap v159: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:41:16.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:16 smithi157 ceph-mon[32580]: pgmap v159: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:41:16.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:16 smithi132 ceph-mon[29178]: pgmap v159: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:41:18.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:18 smithi148 ceph-mon[32311]: pgmap v160: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:41:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:18 smithi157 ceph-mon[32580]: pgmap v160: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:41:18.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:18 smithi132 ceph-mon[29178]: pgmap v160: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:41:20.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:20 smithi148 ceph-mon[32311]: pgmap v161: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-23T18:41:20.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:20 smithi157 ceph-mon[32580]: pgmap v161: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-23T18:41:20.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:20 smithi132 ceph-mon[29178]: pgmap v161: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-23T18:41:22.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:22 smithi148 ceph-mon[32311]: pgmap v162: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:41:22.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:22 smithi157 ceph-mon[32580]: pgmap v162: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:41:22.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:22 smithi132 ceph-mon[29178]: pgmap v162: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:41:24.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:24 smithi148 ceph-mon[32311]: pgmap v163: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:41:24.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:24 smithi157 ceph-mon[32580]: pgmap v163: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:41:24.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:24 smithi132 ceph-mon[29178]: pgmap v163: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:41:26.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:26 smithi148 ceph-mon[32311]: pgmap v164: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:41:26.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:26 smithi157 ceph-mon[32580]: pgmap v164: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:41:26.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:26 smithi132 ceph-mon[29178]: pgmap v164: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:41:28.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:28 smithi148 ceph-mon[32311]: pgmap v165: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:41:28.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:28 smithi157 ceph-mon[32580]: pgmap v165: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:41:28.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:28 smithi132 ceph-mon[29178]: pgmap v165: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:41:30.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:30 smithi148 ceph-mon[32311]: pgmap v166: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:30.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:30 smithi157 ceph-mon[32580]: pgmap v166: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:30.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:30 smithi132 ceph-mon[29178]: pgmap v166: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:32.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:32 smithi148 ceph-mon[32311]: pgmap v167: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:32.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:32 smithi157 ceph-mon[32580]: pgmap v167: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:32.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:32 smithi132 ceph-mon[29178]: pgmap v167: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:34.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:34 smithi148 ceph-mon[32311]: pgmap v168: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:34.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:34 smithi157 ceph-mon[32580]: pgmap v168: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:34.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:34 smithi132 ceph-mon[29178]: pgmap v168: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:36.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:36 smithi148 ceph-mon[32311]: pgmap v169: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:36.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:36 smithi157 ceph-mon[32580]: pgmap v169: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:36.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:36 smithi132 ceph-mon[29178]: pgmap v169: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:38.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:38 smithi148 ceph-mon[32311]: pgmap v170: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:38.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:38 smithi157 ceph-mon[32580]: pgmap v170: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:38.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:38 smithi132 ceph-mon[29178]: pgmap v170: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:40.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:40 smithi148 ceph-mon[32311]: pgmap v171: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:40.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:40 smithi157 ceph-mon[32580]: pgmap v171: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:40.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:40 smithi132 ceph-mon[29178]: pgmap v171: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:42.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:42 smithi148 ceph-mon[32311]: pgmap v172: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:42.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:42 smithi157 ceph-mon[32580]: pgmap v172: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:42.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:42 smithi132 ceph-mon[29178]: pgmap v172: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:44.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:44 smithi148 ceph-mon[32311]: pgmap v173: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:44.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:44 smithi157 ceph-mon[32580]: pgmap v173: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:44.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:44 smithi132 ceph-mon[29178]: pgmap v173: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:46.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:46 smithi148 ceph-mon[32311]: pgmap v174: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:46.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:46 smithi157 ceph-mon[32580]: pgmap v174: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:46.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:46 smithi132 ceph-mon[29178]: pgmap v174: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:48.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:48 smithi157 ceph-mon[32580]: pgmap v175: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:48.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:48 smithi132 ceph-mon[29178]: pgmap v175: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:49.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:48 smithi148 ceph-mon[32311]: pgmap v175: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:50.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:50 smithi157 ceph-mon[32580]: pgmap v176: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:50.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:50 smithi132 ceph-mon[29178]: pgmap v176: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:51.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:50 smithi148 ceph-mon[32311]: pgmap v176: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:52.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:52 smithi157 ceph-mon[32580]: pgmap v177: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:52.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:52 smithi132 ceph-mon[29178]: pgmap v177: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:53.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:52 smithi148 ceph-mon[32311]: pgmap v177: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:54.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:54 smithi157 ceph-mon[32580]: pgmap v178: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:54.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:54 smithi132 ceph-mon[29178]: pgmap v178: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:55.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:54 smithi148 ceph-mon[32311]: pgmap v178: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:56.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:56 smithi157 ceph-mon[32580]: pgmap v179: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:56.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:56 smithi132 ceph-mon[29178]: pgmap v179: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:57.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:56 smithi148 ceph-mon[32311]: pgmap v179: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:41:58.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:41:58 smithi157 ceph-mon[32580]: pgmap v180: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:58.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:41:58 smithi132 ceph-mon[29178]: pgmap v180: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:41:59.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:41:58 smithi148 ceph-mon[32311]: pgmap v180: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:00.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:00 smithi157 ceph-mon[32580]: pgmap v181: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:00.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:00 smithi132 ceph-mon[29178]: pgmap v181: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:01.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:00 smithi148 ceph-mon[32311]: pgmap v181: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:02.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:02 smithi157 ceph-mon[32580]: pgmap v182: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:02.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:02 smithi132 ceph-mon[29178]: pgmap v182: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:03.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:02 smithi148 ceph-mon[32311]: pgmap v182: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:04.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:04 smithi132 ceph-mon[29178]: pgmap v183: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:05.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:04 smithi148 ceph-mon[32311]: pgmap v183: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:05.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:04 smithi157 ceph-mon[32580]: pgmap v183: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:06.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:06 smithi132 ceph-mon[29178]: pgmap v184: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:42:07.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:06 smithi148 ceph-mon[32311]: pgmap v184: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:42:07.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:06 smithi157 ceph-mon[32580]: pgmap v184: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:42:09.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:08 smithi148 ceph-mon[32311]: pgmap v185: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:42:09.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:08 smithi157 ceph-mon[32580]: pgmap v185: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:42:09.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:08 smithi132 ceph-mon[29178]: pgmap v185: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:42:11.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:10 smithi148 ceph-mon[32311]: pgmap v186: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:42:11.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:10 smithi157 ceph-mon[32580]: pgmap v186: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:42:11.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:10 smithi132 ceph-mon[29178]: pgmap v186: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:42:13.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:12 smithi148 ceph-mon[32311]: pgmap v187: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:42:13.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:12 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:42:13.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:12 smithi157 ceph-mon[32580]: pgmap v187: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:42:13.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:12 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:42:13.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:12 smithi132 ceph-mon[29178]: pgmap v187: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:42:13.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:12 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:42:14.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:13 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:42:14.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:13 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:42:14.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:13 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:42:14.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:13 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:42:14.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:13 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:42:14.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:13 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:42:14.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:13 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:42:14.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:13 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:42:14.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:13 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:42:15.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:14 smithi148 ceph-mon[32311]: pgmap v188: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:42:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:14 smithi157 ceph-mon[32580]: pgmap v188: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:42:15.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:14 smithi132 ceph-mon[29178]: pgmap v188: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-23T18:42:17.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:16 smithi148 ceph-mon[32311]: pgmap v189: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:42:17.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:16 smithi157 ceph-mon[32580]: pgmap v189: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:42:17.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:16 smithi132 ceph-mon[29178]: pgmap v189: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T18:42:19.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:18 smithi148 ceph-mon[32311]: pgmap v190: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:42:19.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:18 smithi157 ceph-mon[32580]: pgmap v190: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:42:19.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:18 smithi132 ceph-mon[29178]: pgmap v190: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T18:42:21.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:20 smithi148 ceph-mon[32311]: pgmap v191: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-23T18:42:21.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:20 smithi157 ceph-mon[32580]: pgmap v191: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-23T18:42:21.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:20 smithi132 ceph-mon[29178]: pgmap v191: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-23T18:42:23.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:22 smithi148 ceph-mon[32311]: pgmap v192: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:42:23.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:22 smithi157 ceph-mon[32580]: pgmap v192: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:42:23.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:22 smithi132 ceph-mon[29178]: pgmap v192: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:42:25.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:24 smithi148 ceph-mon[32311]: pgmap v193: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:42:25.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:24 smithi157 ceph-mon[32580]: pgmap v193: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:42:25.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:24 smithi132 ceph-mon[29178]: pgmap v193: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-23T18:42:27.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:26 smithi148 ceph-mon[32311]: pgmap v194: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:42:27.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:26 smithi157 ceph-mon[32580]: pgmap v194: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:42:27.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:26 smithi132 ceph-mon[29178]: pgmap v194: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:42:28.913 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:28 smithi132 ceph-mon[29178]: pgmap v195: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:29.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:28 smithi148 ceph-mon[32311]: pgmap v195: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:29.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:28 smithi157 ceph-mon[32580]: pgmap v195: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:31.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:30 smithi148 ceph-mon[32311]: pgmap v196: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:31.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:30 smithi157 ceph-mon[32580]: pgmap v196: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:31.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:30 smithi132 ceph-mon[29178]: pgmap v196: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:33.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:32 smithi148 ceph-mon[32311]: pgmap v197: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:33.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:32 smithi157 ceph-mon[32580]: pgmap v197: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:33.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:32 smithi132 ceph-mon[29178]: pgmap v197: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:35.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:34 smithi148 ceph-mon[32311]: pgmap v198: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:35.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:34 smithi157 ceph-mon[32580]: pgmap v198: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:35.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:34 smithi132 ceph-mon[29178]: pgmap v198: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:37.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:36 smithi148 ceph-mon[32311]: pgmap v199: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:37.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:36 smithi157 ceph-mon[32580]: pgmap v199: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:37.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:36 smithi132 ceph-mon[29178]: pgmap v199: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:39.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:38 smithi148 ceph-mon[32311]: pgmap v200: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:39.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:38 smithi157 ceph-mon[32580]: pgmap v200: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:39.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:38 smithi132 ceph-mon[29178]: pgmap v200: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:41.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:40 smithi148 ceph-mon[32311]: pgmap v201: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:41.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:40 smithi157 ceph-mon[32580]: pgmap v201: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:41.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:40 smithi132 ceph-mon[29178]: pgmap v201: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:43.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:42 smithi148 ceph-mon[32311]: pgmap v202: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:43.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:42 smithi157 ceph-mon[32580]: pgmap v202: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:43.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:42 smithi132 ceph-mon[29178]: pgmap v202: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:45.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:44 smithi148 ceph-mon[32311]: pgmap v203: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:45.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:44 smithi157 ceph-mon[32580]: pgmap v203: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:45.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:44 smithi132 ceph-mon[29178]: pgmap v203: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:47.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:46 smithi148 ceph-mon[32311]: pgmap v204: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:47.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:46 smithi157 ceph-mon[32580]: pgmap v204: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:47.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:46 smithi132 ceph-mon[29178]: pgmap v204: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:49.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:48 smithi148 ceph-mon[32311]: pgmap v205: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:48 smithi157 ceph-mon[32580]: pgmap v205: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:48 smithi132 ceph-mon[29178]: pgmap v205: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:51.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:50 smithi148 ceph-mon[32311]: pgmap v206: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:51.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:50 smithi157 ceph-mon[32580]: pgmap v206: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:51.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:50 smithi132 ceph-mon[29178]: pgmap v206: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:52.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:51 smithi148 ceph-mon[32311]: pgmap v207: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:52.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:51 smithi157 ceph-mon[32580]: pgmap v207: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:52.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:51 smithi132 ceph-mon[29178]: pgmap v207: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:54.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:53 smithi148 ceph-mon[32311]: pgmap v208: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:54.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:53 smithi157 ceph-mon[32580]: pgmap v208: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:54.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:53 smithi132 ceph-mon[29178]: pgmap v208: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:56.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:55 smithi148 ceph-mon[32311]: pgmap v209: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:56.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:55 smithi157 ceph-mon[32580]: pgmap v209: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:56.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:55 smithi132 ceph-mon[29178]: pgmap v209: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:42:58.163 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:57 smithi132 ceph-mon[29178]: pgmap v210: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:58.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:57 smithi148 ceph-mon[32311]: pgmap v210: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:42:58.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:57 smithi157 ceph-mon[32580]: pgmap v210: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:00.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:42:59 smithi148 ceph-mon[32311]: pgmap v211: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:00.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:42:59 smithi157 ceph-mon[32580]: pgmap v211: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:00.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:42:59 smithi132 ceph-mon[29178]: pgmap v211: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:02.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:01 smithi148 ceph-mon[32311]: pgmap v212: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:02.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:01 smithi157 ceph-mon[32580]: pgmap v212: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:02.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:01 smithi132 ceph-mon[29178]: pgmap v212: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:04.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:03 smithi148 ceph-mon[32311]: pgmap v213: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:04.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:03 smithi157 ceph-mon[32580]: pgmap v213: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:04.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:03 smithi132 ceph-mon[29178]: pgmap v213: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:06.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:05 smithi148 ceph-mon[32311]: pgmap v214: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:43:06.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:05 smithi157 ceph-mon[32580]: pgmap v214: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:43:06.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:05 smithi132 ceph-mon[29178]: pgmap v214: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:43:08.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:07 smithi148 ceph-mon[32311]: pgmap v215: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T18:43:08.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:07 smithi157 ceph-mon[32580]: pgmap v215: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T18:43:08.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:07 smithi132 ceph-mon[29178]: pgmap v215: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T18:43:10.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:09 smithi148 ceph-mon[32311]: pgmap v216: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 31 op/s 2024-04-23T18:43:10.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:09 smithi157 ceph-mon[32580]: pgmap v216: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 31 op/s 2024-04-23T18:43:10.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:09 smithi132 ceph-mon[29178]: pgmap v216: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 31 op/s 2024-04-23T18:43:11.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:10 smithi148 ceph-mon[32311]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T18:43:11.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:10 smithi157 ceph-mon[32580]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T18:43:11.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:10 smithi132 ceph-mon[29178]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T18:43:12.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:11 smithi148 ceph-mon[32311]: pgmap v218: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:43:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:11 smithi157 ceph-mon[32580]: pgmap v218: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:43:12.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:11 smithi132 ceph-mon[29178]: pgmap v218: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:43:14.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:13 smithi148 ceph-mon[32311]: pgmap v219: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:43:14.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:13 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:43:14.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:13 smithi157 ceph-mon[32580]: pgmap v219: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:43:14.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:13 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:43:14.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:13 smithi132 ceph-mon[29178]: pgmap v219: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:43:14.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:13 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:43:15.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:14 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:43:15.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:14 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:43:15.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:14 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:43:15.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:14 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:43:15.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:14 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:43:15.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:14 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:43:15.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:14 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:43:15.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:14 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:43:15.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:14 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:43:17.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:16 smithi148 ceph-mon[32311]: pgmap v220: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:43:17.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:16 smithi148 ceph-mon[32311]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T18:43:17.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:16 smithi157 ceph-mon[32580]: pgmap v220: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:43:17.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:16 smithi157 ceph-mon[32580]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T18:43:17.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:16 smithi132 ceph-mon[29178]: pgmap v220: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:43:17.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:16 smithi132 ceph-mon[29178]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T18:43:19.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:18 smithi148 ceph-mon[32311]: pgmap v222: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-23T18:43:19.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:18 smithi157 ceph-mon[32580]: pgmap v222: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-23T18:43:19.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:18 smithi132 ceph-mon[29178]: pgmap v222: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-23T18:43:21.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:20 smithi148 ceph-mon[32311]: pgmap v223: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 12 op/s 2024-04-23T18:43:21.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:20 smithi157 ceph-mon[32580]: pgmap v223: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 12 op/s 2024-04-23T18:43:21.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:20 smithi132 ceph-mon[29178]: pgmap v223: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 12 op/s 2024-04-23T18:43:23.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:22 smithi148 ceph-mon[32311]: pgmap v224: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:43:23.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:22 smithi157 ceph-mon[32580]: pgmap v224: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:43:23.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:22 smithi132 ceph-mon[29178]: pgmap v224: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:43:25.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:24 smithi148 ceph-mon[32311]: pgmap v225: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:43:25.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:24 smithi157 ceph-mon[32580]: pgmap v225: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:43:25.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:24 smithi132 ceph-mon[29178]: pgmap v225: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:43:27.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:26 smithi148 ceph-mon[32311]: pgmap v226: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:43:27.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:26 smithi157 ceph-mon[32580]: pgmap v226: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:43:27.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:26 smithi132 ceph-mon[29178]: pgmap v226: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:43:28.913 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:28 smithi132 ceph-mon[29178]: pgmap v227: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-04-23T18:43:29.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:28 smithi148 ceph-mon[32311]: pgmap v227: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-04-23T18:43:29.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:28 smithi157 ceph-mon[32580]: pgmap v227: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-04-23T18:43:31.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:30 smithi148 ceph-mon[32311]: pgmap v228: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:43:31.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:30 smithi157 ceph-mon[32580]: pgmap v228: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:43:31.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:30 smithi132 ceph-mon[29178]: pgmap v228: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:43:33.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:32 smithi148 ceph-mon[32311]: pgmap v229: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:33.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:32 smithi157 ceph-mon[32580]: pgmap v229: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:33.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:32 smithi132 ceph-mon[29178]: pgmap v229: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:35.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:34 smithi148 ceph-mon[32311]: pgmap v230: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:35.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:34 smithi157 ceph-mon[32580]: pgmap v230: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:35.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:34 smithi132 ceph-mon[29178]: pgmap v230: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:37.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:36 smithi148 ceph-mon[32311]: pgmap v231: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:37.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:36 smithi157 ceph-mon[32580]: pgmap v231: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:37.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:36 smithi132 ceph-mon[29178]: pgmap v231: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:39.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:38 smithi148 ceph-mon[32311]: pgmap v232: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:39.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:38 smithi157 ceph-mon[32580]: pgmap v232: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:39.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:38 smithi132 ceph-mon[29178]: pgmap v232: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:41.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:40 smithi148 ceph-mon[32311]: pgmap v233: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:41.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:40 smithi157 ceph-mon[32580]: pgmap v233: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:41.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:40 smithi132 ceph-mon[29178]: pgmap v233: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:43.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:42 smithi148 ceph-mon[32311]: pgmap v234: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:43.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:42 smithi157 ceph-mon[32580]: pgmap v234: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:43.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:42 smithi132 ceph-mon[29178]: pgmap v234: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:45.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:44 smithi148 ceph-mon[32311]: pgmap v235: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:45.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:44 smithi157 ceph-mon[32580]: pgmap v235: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:45.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:44 smithi132 ceph-mon[29178]: pgmap v235: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:47.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:46 smithi148 ceph-mon[32311]: pgmap v236: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:47.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:46 smithi157 ceph-mon[32580]: pgmap v236: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:47.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:46 smithi132 ceph-mon[29178]: pgmap v236: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:49.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:48 smithi148 ceph-mon[32311]: pgmap v237: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:48 smithi157 ceph-mon[32580]: pgmap v237: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:49.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:48 smithi132 ceph-mon[29178]: pgmap v237: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:51.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:50 smithi148 ceph-mon[32311]: pgmap v238: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:51.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:50 smithi157 ceph-mon[32580]: pgmap v238: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:51.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:50 smithi132 ceph-mon[29178]: pgmap v238: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:53.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:52 smithi148 ceph-mon[32311]: pgmap v239: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:53.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:52 smithi157 ceph-mon[32580]: pgmap v239: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:53.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:52 smithi132 ceph-mon[29178]: pgmap v239: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:55.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:54 smithi148 ceph-mon[32311]: pgmap v240: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:55.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:54 smithi157 ceph-mon[32580]: pgmap v240: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:55.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:54 smithi132 ceph-mon[29178]: pgmap v240: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:57.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:56 smithi148 ceph-mon[32311]: pgmap v241: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:57.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:56 smithi157 ceph-mon[32580]: pgmap v241: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:57.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:56 smithi132 ceph-mon[29178]: pgmap v241: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:43:59.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:43:58 smithi148 ceph-mon[32311]: pgmap v242: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:59.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:43:58 smithi157 ceph-mon[32580]: pgmap v242: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:43:59.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:43:58 smithi132 ceph-mon[29178]: pgmap v242: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:01.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:00 smithi148 ceph-mon[32311]: pgmap v243: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:01.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:00 smithi157 ceph-mon[32580]: pgmap v243: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:01.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:00 smithi132 ceph-mon[29178]: pgmap v243: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:03.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:02 smithi148 ceph-mon[32311]: pgmap v244: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:03.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:02 smithi157 ceph-mon[32580]: pgmap v244: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:03.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:02 smithi132 ceph-mon[29178]: pgmap v244: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:04.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:03 smithi148 ceph-mon[32311]: pgmap v245: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:04.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:03 smithi157 ceph-mon[32580]: pgmap v245: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:04.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:03 smithi132 ceph-mon[29178]: pgmap v245: 97 pgs: 97 active+clean; 9.8 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:06.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:05 smithi148 ceph-mon[32311]: pgmap v246: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:44:06.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:05 smithi157 ceph-mon[32580]: pgmap v246: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:44:06.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:05 smithi132 ceph-mon[29178]: pgmap v246: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:44:08.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:07 smithi148 ceph-mon[32311]: pgmap v247: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T18:44:08.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:07 smithi157 ceph-mon[32580]: pgmap v247: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T18:44:08.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:07 smithi132 ceph-mon[29178]: pgmap v247: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T18:44:10.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:09 smithi148 ceph-mon[32311]: pgmap v248: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 31 op/s 2024-04-23T18:44:10.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:09 smithi157 ceph-mon[32580]: pgmap v248: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 31 op/s 2024-04-23T18:44:10.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:09 smithi132 ceph-mon[29178]: pgmap v248: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 31 op/s 2024-04-23T18:44:11.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:10 smithi148 ceph-mon[32311]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T18:44:11.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:10 smithi157 ceph-mon[32580]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T18:44:11.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:10 smithi132 ceph-mon[29178]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T18:44:12.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:11 smithi148 ceph-mon[32311]: pgmap v250: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:44:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:11 smithi157 ceph-mon[32580]: pgmap v250: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:44:12.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:11 smithi132 ceph-mon[29178]: pgmap v250: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:44:14.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:13 smithi148 ceph-mon[32311]: pgmap v251: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:44:14.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:13 smithi157 ceph-mon[32580]: pgmap v251: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:44:14.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:13 smithi132 ceph-mon[29178]: pgmap v251: 97 pgs: 97 active+clean; 9.8 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:44:15.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:14 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:44:15.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:14 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:44:15.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:14 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:44:15.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:14 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:44:15.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:14 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:44:15.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:14 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:44:15.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:14 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:44:15.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:14 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:44:15.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:14 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:44:15.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:14 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:44:15.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:14 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:44:15.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:14 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:44:16.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:15 smithi148 ceph-mon[32311]: pgmap v252: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:16.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:15 smithi157 ceph-mon[32580]: pgmap v252: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:16.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:15 smithi132 ceph-mon[29178]: pgmap v252: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:18.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:17 smithi148 ceph-mon[32311]: pgmap v253: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:18.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:17 smithi157 ceph-mon[32580]: pgmap v253: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:18.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:17 smithi132 ceph-mon[29178]: pgmap v253: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:20.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:19 smithi148 ceph-mon[32311]: pgmap v254: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:20.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:19 smithi157 ceph-mon[32580]: pgmap v254: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:20.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:19 smithi132 ceph-mon[29178]: pgmap v254: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:22.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:21 smithi148 ceph-mon[32311]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T18:44:22.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:21 smithi157 ceph-mon[32580]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T18:44:22.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:21 smithi132 ceph-mon[29178]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T18:44:23.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:22 smithi148 ceph-mon[32311]: pgmap v256: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:23.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:22 smithi157 ceph-mon[32580]: pgmap v256: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:23.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:22 smithi132 ceph-mon[29178]: pgmap v256: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:25.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:24 smithi148 ceph-mon[32311]: pgmap v257: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:25.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:24 smithi157 ceph-mon[32580]: pgmap v257: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:25.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:24 smithi132 ceph-mon[29178]: pgmap v257: 97 pgs: 97 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:44:27.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:26 smithi148 ceph-mon[32311]: pgmap v258: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:44:27.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:26 smithi157 ceph-mon[32580]: pgmap v258: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:44:27.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:26 smithi132 ceph-mon[29178]: pgmap v258: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:44:28.913 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:28 smithi132 ceph-mon[29178]: pgmap v259: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:44:29.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:28 smithi148 ceph-mon[32311]: pgmap v259: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:44:29.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:28 smithi157 ceph-mon[32580]: pgmap v259: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:44:31.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:30 smithi148 ceph-mon[32311]: pgmap v260: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-23T18:44:31.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:30 smithi157 ceph-mon[32580]: pgmap v260: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-23T18:44:31.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:30 smithi132 ceph-mon[29178]: pgmap v260: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-23T18:44:33.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:32 smithi148 ceph-mon[32311]: pgmap v261: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-04-23T18:44:33.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:32 smithi157 ceph-mon[32580]: pgmap v261: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-04-23T18:44:33.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:32 smithi132 ceph-mon[29178]: pgmap v261: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-04-23T18:44:35.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:34 smithi148 ceph-mon[32311]: pgmap v262: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:35.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:34 smithi157 ceph-mon[32580]: pgmap v262: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:35.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:34 smithi132 ceph-mon[29178]: pgmap v262: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:37.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:36 smithi148 ceph-mon[32311]: pgmap v263: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:37.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:36 smithi157 ceph-mon[32580]: pgmap v263: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:37.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:36 smithi132 ceph-mon[29178]: pgmap v263: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:39.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:38 smithi148 ceph-mon[32311]: pgmap v264: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:39.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:38 smithi157 ceph-mon[32580]: pgmap v264: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:39.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:38 smithi132 ceph-mon[29178]: pgmap v264: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:41.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:40 smithi148 ceph-mon[32311]: pgmap v265: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:41.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:40 smithi157 ceph-mon[32580]: pgmap v265: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:41.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:40 smithi132 ceph-mon[29178]: pgmap v265: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:43.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:42 smithi148 ceph-mon[32311]: pgmap v266: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:43.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:42 smithi157 ceph-mon[32580]: pgmap v266: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:43.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:42 smithi132 ceph-mon[29178]: pgmap v266: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:45.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:44 smithi148 ceph-mon[32311]: pgmap v267: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:45.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:44 smithi157 ceph-mon[32580]: pgmap v267: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:45.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:44 smithi132 ceph-mon[29178]: pgmap v267: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:47.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:46 smithi148 ceph-mon[32311]: pgmap v268: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:47.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:46 smithi157 ceph-mon[32580]: pgmap v268: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:47.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:46 smithi132 ceph-mon[29178]: pgmap v268: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:49.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:48 smithi148 ceph-mon[32311]: pgmap v269: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:49.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:48 smithi157 ceph-mon[32580]: pgmap v269: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:49.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:48 smithi132 ceph-mon[29178]: pgmap v269: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:51.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:50 smithi148 ceph-mon[32311]: pgmap v270: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:51.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:50 smithi157 ceph-mon[32580]: pgmap v270: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:51.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:50 smithi132 ceph-mon[29178]: pgmap v270: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:53.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:52 smithi148 ceph-mon[32311]: pgmap v271: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:53.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:52 smithi157 ceph-mon[32580]: pgmap v271: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:53.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:52 smithi132 ceph-mon[29178]: pgmap v271: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:55.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:54 smithi148 ceph-mon[32311]: pgmap v272: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:55.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:54 smithi157 ceph-mon[32580]: pgmap v272: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:55.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:54 smithi132 ceph-mon[29178]: pgmap v272: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:57.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:56 smithi148 ceph-mon[32311]: pgmap v273: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:57.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:56 smithi157 ceph-mon[32580]: pgmap v273: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:57.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:56 smithi132 ceph-mon[29178]: pgmap v273: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:44:59.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:44:58 smithi148 ceph-mon[32311]: pgmap v274: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:59.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:44:58 smithi157 ceph-mon[32580]: pgmap v274: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:44:59.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:44:58 smithi132 ceph-mon[29178]: pgmap v274: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:01.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:00 smithi148 ceph-mon[32311]: pgmap v275: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:01.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:00 smithi157 ceph-mon[32580]: pgmap v275: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:01.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:00 smithi132 ceph-mon[29178]: pgmap v275: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:02.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:01 smithi148 ceph-mon[32311]: pgmap v276: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:02.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:01 smithi157 ceph-mon[32580]: pgmap v276: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:02.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:01 smithi132 ceph-mon[29178]: pgmap v276: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:04.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:03 smithi148 ceph-mon[32311]: pgmap v277: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:04.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:03 smithi157 ceph-mon[32580]: pgmap v277: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:04.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:03 smithi132 ceph-mon[29178]: pgmap v277: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:06.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:05 smithi148 ceph-mon[32311]: pgmap v278: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:45:06.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:05 smithi157 ceph-mon[32580]: pgmap v278: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:45:06.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:05 smithi132 ceph-mon[29178]: pgmap v278: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:45:08.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:07 smithi148 ceph-mon[32311]: pgmap v279: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T18:45:08.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:07 smithi157 ceph-mon[32580]: pgmap v279: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T18:45:08.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:07 smithi132 ceph-mon[29178]: pgmap v279: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T18:45:10.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:09 smithi148 ceph-mon[32311]: pgmap v280: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 31 op/s 2024-04-23T18:45:10.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:09 smithi157 ceph-mon[32580]: pgmap v280: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 31 op/s 2024-04-23T18:45:10.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:09 smithi132 ceph-mon[29178]: pgmap v280: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 31 op/s 2024-04-23T18:45:11.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:10 smithi148 ceph-mon[32311]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T18:45:11.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:10 smithi157 ceph-mon[32580]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T18:45:11.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:10 smithi132 ceph-mon[29178]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T18:45:12.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:11 smithi148 ceph-mon[32311]: pgmap v282: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:45:12.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:11 smithi157 ceph-mon[32580]: pgmap v282: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:45:12.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:11 smithi132 ceph-mon[29178]: pgmap v282: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:45:14.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:13 smithi148 ceph-mon[32311]: pgmap v283: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:45:14.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:13 smithi157 ceph-mon[32580]: pgmap v283: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:45:14.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:13 smithi132 ceph-mon[29178]: pgmap v283: 97 pgs: 97 active+clean; 9.8 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:45:15.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:14 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:45:15.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:14 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:45:15.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:14 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:45:16.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:15 smithi148 ceph-mon[32311]: pgmap v284: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:45:16.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:15 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:45:16.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:15 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:45:16.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:15 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:45:16.264 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:15 smithi148 ceph-mon[32311]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T18:45:16.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:15 smithi157 ceph-mon[32580]: pgmap v284: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:45:16.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:15 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:45:16.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:15 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:45:16.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:15 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:45:16.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:15 smithi157 ceph-mon[32580]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T18:45:16.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:15 smithi132 ceph-mon[29178]: pgmap v284: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:45:16.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:15 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:45:16.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:15 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:45:16.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:15 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:45:16.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:15 smithi132 ceph-mon[29178]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T18:45:18.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:17 smithi148 ceph-mon[32311]: pgmap v286: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-23T18:45:18.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:17 smithi157 ceph-mon[32580]: pgmap v286: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-23T18:45:18.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:17 smithi132 ceph-mon[29178]: pgmap v286: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-23T18:45:20.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:19 smithi148 ceph-mon[32311]: pgmap v287: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 12 op/s 2024-04-23T18:45:20.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:19 smithi157 ceph-mon[32580]: pgmap v287: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 12 op/s 2024-04-23T18:45:20.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:19 smithi132 ceph-mon[29178]: pgmap v287: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 12 op/s 2024-04-23T18:45:22.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:21 smithi148 ceph-mon[32311]: pgmap v288: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:45:22.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:21 smithi157 ceph-mon[32580]: pgmap v288: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:45:22.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:21 smithi132 ceph-mon[29178]: pgmap v288: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:45:24.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:23 smithi148 ceph-mon[32311]: pgmap v289: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:45:24.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:23 smithi157 ceph-mon[32580]: pgmap v289: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:45:24.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:23 smithi132 ceph-mon[29178]: pgmap v289: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:45:26.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:25 smithi148 ceph-mon[32311]: pgmap v290: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:45:26.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:25 smithi157 ceph-mon[32580]: pgmap v290: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:45:26.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:25 smithi132 ceph-mon[29178]: pgmap v290: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:45:28.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:27 smithi148 ceph-mon[32311]: pgmap v291: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-04-23T18:45:28.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:27 smithi157 ceph-mon[32580]: pgmap v291: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-04-23T18:45:28.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:27 smithi132 ceph-mon[29178]: pgmap v291: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2024-04-23T18:45:30.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:29 smithi148 ceph-mon[32311]: pgmap v292: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:45:30.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:29 smithi157 ceph-mon[32580]: pgmap v292: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:45:30.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:29 smithi132 ceph-mon[29178]: pgmap v292: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-23T18:45:32.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:31 smithi148 ceph-mon[32311]: pgmap v293: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:32.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:31 smithi157 ceph-mon[32580]: pgmap v293: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:32.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:31 smithi132 ceph-mon[29178]: pgmap v293: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:34.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:34 smithi148 ceph-mon[32311]: pgmap v294: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:34.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:34 smithi157 ceph-mon[32580]: pgmap v294: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:34.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:34 smithi132 ceph-mon[29178]: pgmap v294: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:36.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:36 smithi148 ceph-mon[32311]: pgmap v295: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:36.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:36 smithi157 ceph-mon[32580]: pgmap v295: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:36.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:36 smithi132 ceph-mon[29178]: pgmap v295: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:38.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:38 smithi148 ceph-mon[32311]: pgmap v296: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:38.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:38 smithi157 ceph-mon[32580]: pgmap v296: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:38.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:38 smithi132 ceph-mon[29178]: pgmap v296: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:40.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:40 smithi148 ceph-mon[32311]: pgmap v297: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:40.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:40 smithi157 ceph-mon[32580]: pgmap v297: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:40.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:40 smithi132 ceph-mon[29178]: pgmap v297: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:42.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:42 smithi148 ceph-mon[32311]: pgmap v298: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:42.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:42 smithi157 ceph-mon[32580]: pgmap v298: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:42.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:42 smithi132 ceph-mon[29178]: pgmap v298: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:44.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:44 smithi148 ceph-mon[32311]: pgmap v299: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:44.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:44 smithi157 ceph-mon[32580]: pgmap v299: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:44.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:44 smithi132 ceph-mon[29178]: pgmap v299: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:46.263 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:46 smithi148 ceph-mon[32311]: pgmap v300: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:46.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:46 smithi157 ceph-mon[32580]: pgmap v300: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:46.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:46 smithi132 ceph-mon[29178]: pgmap v300: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:48.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:48 smithi157 ceph-mon[32580]: pgmap v301: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:48.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:48 smithi132 ceph-mon[29178]: pgmap v301: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:48.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:48 smithi148 ceph-mon[32311]: pgmap v301: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:50.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:50 smithi157 ceph-mon[32580]: pgmap v302: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:50.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:50 smithi132 ceph-mon[29178]: pgmap v302: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:50.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:50 smithi148 ceph-mon[32311]: pgmap v302: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:52.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:52 smithi157 ceph-mon[32580]: pgmap v303: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:52.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:52 smithi132 ceph-mon[29178]: pgmap v303: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:52.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:52 smithi148 ceph-mon[32311]: pgmap v303: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:54.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:54 smithi157 ceph-mon[32580]: pgmap v304: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:54.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:54 smithi132 ceph-mon[29178]: pgmap v304: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:54.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:54 smithi148 ceph-mon[32311]: pgmap v304: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:56.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:56 smithi157 ceph-mon[32580]: pgmap v305: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:56.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:56 smithi132 ceph-mon[29178]: pgmap v305: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:56.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:56 smithi148 ceph-mon[32311]: pgmap v305: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:45:58.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:45:58 smithi157 ceph-mon[32580]: pgmap v306: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:58.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:45:58 smithi132 ceph-mon[29178]: pgmap v306: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:45:58.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:45:58 smithi148 ceph-mon[32311]: pgmap v306: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:00.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:00 smithi157 ceph-mon[32580]: pgmap v307: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:46:00.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:00 smithi132 ceph-mon[29178]: pgmap v307: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:46:00.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:00 smithi148 ceph-mon[32311]: pgmap v307: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:46:02.289 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:02 smithi157 ceph-mon[32580]: pgmap v308: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:02.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:02 smithi132 ceph-mon[29178]: pgmap v308: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:02.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:02 smithi148 ceph-mon[32311]: pgmap v308: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:04.288 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:04 smithi157 ceph-mon[32580]: pgmap v309: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:04.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:04 smithi132 ceph-mon[29178]: pgmap v309: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:04.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:04 smithi148 ceph-mon[32311]: pgmap v309: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:05.675 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-23T18:46:05.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.796+0000 7f22f4f20640 1 -- 172.21.15.132:0/1160605470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22f01010f0 msgr2=0x7f22f01014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:05.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.796+0000 7f22f4f20640 1 --2- 172.21.15.132:0/1160605470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22f01010f0 0x7f22f01014d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009900 tx=0x7f22e002a410 comp rx=0 tx=0).stop 2024-04-23T18:46:05.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.796+0000 7f22f4f20640 1 -- 172.21.15.132:0/1160605470 shutdown_connections 2024-04-23T18:46:05.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.796+0000 7f22f4f20640 1 --2- 172.21.15.132:0/1160605470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0106170 0x7f22f010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:05.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.796+0000 7f22f4f20640 1 --2- 172.21.15.132:0/1160605470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0101aa0 0x7f22f0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:05.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.796+0000 7f22f4f20640 1 --2- 172.21.15.132:0/1160605470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22f01010f0 0x7f22f01014d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:05.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.796+0000 7f22f4f20640 1 -- 172.21.15.132:0/1160605470 >> 172.21.15.132:0/1160605470 conn(0x7f22f0077fd0 msgr2=0x7f22f00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:05.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.796+0000 7f22f4f20640 1 -- 172.21.15.132:0/1160605470 shutdown_connections 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.796+0000 7f22f4f20640 1 -- 172.21.15.132:0/1160605470 wait complete. 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.797+0000 7f22f4f20640 1 Processor -- start 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.797+0000 7f22f4f20640 1 -- start start 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.797+0000 7f22f4f20640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f01010f0 0x7f22f01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.797+0000 7f22f4f20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0106170 0x7f22f0198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.797+0000 7f22f4f20640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22f019cf80 0x7f22f019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.797+0000 7f22f4f20640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01110b0 con 0x7f22f019cf80 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.797+0000 7f22f4f20640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0110f30 con 0x7f22f01010f0 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.797+0000 7f22f4f20640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0111230 con 0x7f22f0106170 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.797+0000 7f22eed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22f019cf80 0x7f22f019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.797+0000 7f22eed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22f019cf80 0x7f22f019d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:47116/0 (socket says 172.21.15.132:47116) 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.797+0000 7f22eed9d640 1 -- 172.21.15.132:0/1790683556 learned_addr learned my addr 172.21.15.132:0/1790683556 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22e7fff640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0106170 0x7f22f0198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:05.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22ee59c640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f01010f0 0x7f22f01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:05.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22eed9d640 1 -- 172.21.15.132:0/1790683556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0106170 msgr2=0x7f22f0198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:05.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22eed9d640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0106170 0x7f22f0198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:05.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22eed9d640 1 -- 172.21.15.132:0/1790683556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f01010f0 msgr2=0x7f22f01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:05.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22eed9d640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f01010f0 0x7f22f01986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:05.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22eed9d640 1 -- 172.21.15.132:0/1790683556 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f019da70 con 0x7f22f019cf80 2024-04-23T18:46:05.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22eed9d640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22f019cf80 0x7f22f019d430 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f22e800cce0 tx=0x7f22e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:05.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22e77fe640 1 -- 172.21.15.132:0/1790683556 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e800f450 con 0x7f22f019cf80 2024-04-23T18:46:05.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22e77fe640 1 -- 172.21.15.132:0/1790683556 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22e800faa0 con 0x7f22f019cf80 2024-04-23T18:46:05.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22f4f20640 1 -- 172.21.15.132:0/1790683556 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f019dd60 con 0x7f22f019cf80 2024-04-23T18:46:05.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22e77fe640 1 -- 172.21.15.132:0/1790683556 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8018b70 con 0x7f22f019cf80 2024-04-23T18:46:05.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.798+0000 7f22f4f20640 1 -- 172.21.15.132:0/1790683556 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f01a5500 con 0x7f22f019cf80 2024-04-23T18:46:05.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.799+0000 7f22f4f20640 1 -- 172.21.15.132:0/1790683556 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f004fce0 con 0x7f22f019cf80 2024-04-23T18:46:05.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.800+0000 7f22e77fe640 1 -- 172.21.15.132:0/1790683556 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f22e8018d10 con 0x7f22f019cf80 2024-04-23T18:46:05.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.801+0000 7f22e77fe640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f22dc0a3dc0 0x7f22dc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:05.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.801+0000 7f22e77fe640 1 -- 172.21.15.132:0/1790683556 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f22e80d0ce0 con 0x7f22f019cf80 2024-04-23T18:46:05.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.801+0000 7f22ee59c640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f22dc0a3dc0 0x7f22dc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:05.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.801+0000 7f22ee59c640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f22dc0a3dc0 0x7f22dc0a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f22e0031440 tx=0x7f22e00095f0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:05.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:05.805+0000 7f22e77fe640 1 -- 172.21.15.132:0/1790683556 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22e8014030 con 0x7f22f019cf80 2024-04-23T18:46:05.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.007+0000 7f22f4f20640 1 -- 172.21.15.132:0/1790683556 --> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f22f0101aa0 con 0x7f22dc0a3dc0 2024-04-23T18:46:06.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.014+0000 7f22e77fe640 1 -- 172.21.15.132:0/1790683556 <== mgr.15366 v2:172.21.15.132:6800/3428544997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f22f0101aa0 con 0x7f22dc0a3dc0 2024-04-23T18:46:06.005 INFO:teuthology.orchestra.run.smithi132.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-23T00:00:00", "created": "2024-04-23T18:36:04", "first": "2024-04-23T18:37:00", "last": "2024-04-23T18:46:00", "last_pruned": "2024-04-23T18:46:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-23T18:46:06.007 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.016+0000 7f22f4f20640 1 -- 172.21.15.132:0/1790683556 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f22dc0a3dc0 msgr2=0x7f22dc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.016+0000 7f22f4f20640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f22dc0a3dc0 0x7f22dc0a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f22e0031440 tx=0x7f22e00095f0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.016+0000 7f22f4f20640 1 -- 172.21.15.132:0/1790683556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22f019cf80 msgr2=0x7f22f019d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.016+0000 7f22f4f20640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22f019cf80 0x7f22f019d430 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f22e800cce0 tx=0x7f22e800a620 comp rx=0 tx=0).stop 2024-04-23T18:46:06.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.016+0000 7f22f4f20640 1 -- 172.21.15.132:0/1790683556 shutdown_connections 2024-04-23T18:46:06.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.016+0000 7f22f4f20640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f22dc0a3dc0 0x7f22dc0a6280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.016+0000 7f22f4f20640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f0106170 0x7f22f0198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.016+0000 7f22f4f20640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22f019cf80 0x7f22f019d430 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.016+0000 7f22f4f20640 1 --2- 172.21.15.132:0/1790683556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f01010f0 0x7f22f01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.016+0000 7f22f4f20640 1 -- 172.21.15.132:0/1790683556 >> 172.21.15.132:0/1790683556 conn(0x7f22f0077fd0 msgr2=0x7f22f00699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:06.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.016+0000 7f22f4f20640 1 -- 172.21.15.132:0/1790683556 shutdown_connections 2024-04-23T18:46:06.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.016+0000 7f22f4f20640 1 -- 172.21.15.132:0/1790683556 wait complete. 2024-04-23T18:46:06.018 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-23T18:46:06.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.127+0000 7f8ca1074640 1 -- 172.21.15.132:0/2557698556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9c1010f0 msgr2=0x7f8c9c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.127+0000 7f8ca1074640 1 --2- 172.21.15.132:0/2557698556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9c1010f0 0x7f8c9c1014d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009900 tx=0x7f8c9402a410 comp rx=0 tx=0).stop 2024-04-23T18:46:06.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.127+0000 7f8ca1074640 1 -- 172.21.15.132:0/2557698556 shutdown_connections 2024-04-23T18:46:06.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.127+0000 7f8ca1074640 1 --2- 172.21.15.132:0/2557698556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c9c106170 0x7f8c9c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.127+0000 7f8ca1074640 1 --2- 172.21.15.132:0/2557698556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c101aa0 0x7f8c9c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.127+0000 7f8ca1074640 1 --2- 172.21.15.132:0/2557698556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9c1010f0 0x7f8c9c1014d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.127+0000 7f8ca1074640 1 -- 172.21.15.132:0/2557698556 >> 172.21.15.132:0/2557698556 conn(0x7f8c9c077fd0 msgr2=0x7f8c9c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:06.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.127+0000 7f8ca1074640 1 -- 172.21.15.132:0/2557698556 shutdown_connections 2024-04-23T18:46:06.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.127+0000 7f8ca1074640 1 -- 172.21.15.132:0/2557698556 wait complete. 2024-04-23T18:46:06.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8ca1074640 1 Processor -- start 2024-04-23T18:46:06.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8ca1074640 1 -- start start 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8ca1074640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9c1010f0 0x7f8c9c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8ca1074640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c106170 0x7f8c9c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8ca1074640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c9c199f70 0x7f8c9c1a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8ca1074640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c1110e0 con 0x7f8c9c1010f0 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8ca1074640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c110f60 con 0x7f8c9c106170 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8ca1074640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c111260 con 0x7f8c9c199f70 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8c9ad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9c1010f0 0x7f8c9c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8c9ad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9c1010f0 0x7f8c9c1994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:47126/0 (socket says 172.21.15.132:47126) 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8c9ad9d640 1 -- 172.21.15.132:0/1993075313 learned_addr learned my addr 172.21.15.132:0/1993075313 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8c9a59c640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c106170 0x7f8c9c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8c9b59e640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c9c199f70 0x7f8c9c1a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8c9ad9d640 1 -- 172.21.15.132:0/1993075313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c9c199f70 msgr2=0x7f8c9c1a4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8c9ad9d640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c9c199f70 0x7f8c9c1a4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8c9ad9d640 1 -- 172.21.15.132:0/1993075313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c106170 msgr2=0x7f8c9c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8c9ad9d640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c106170 0x7f8c9c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.128+0000 7f8c9ad9d640 1 -- 172.21.15.132:0/1993075313 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9c1a4cf0 con 0x7f8c9c1010f0 2024-04-23T18:46:06.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.129+0000 7f8c9ad9d640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9c1010f0 0x7f8c9c1994a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8c94006010 tx=0x7f8c940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:06.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.129+0000 7f8c83fff640 1 -- 172.21.15.132:0/1993075313 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9403f030 con 0x7f8c9c1010f0 2024-04-23T18:46:06.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.129+0000 7f8c83fff640 1 -- 172.21.15.132:0/1993075313 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c94003800 con 0x7f8c9c1010f0 2024-04-23T18:46:06.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.129+0000 7f8ca1074640 1 -- 172.21.15.132:0/1993075313 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9c1a4f80 con 0x7f8c9c1010f0 2024-04-23T18:46:06.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.129+0000 7f8c83fff640 1 -- 172.21.15.132:0/1993075313 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94035070 con 0x7f8c9c1010f0 2024-04-23T18:46:06.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.129+0000 7f8ca1074640 1 -- 172.21.15.132:0/1993075313 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c9c1a53e0 con 0x7f8c9c1010f0 2024-04-23T18:46:06.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.130+0000 7f8ca1074640 1 -- 172.21.15.132:0/1993075313 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c9c04fce0 con 0x7f8c9c1010f0 2024-04-23T18:46:06.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.131+0000 7f8c83fff640 1 -- 172.21.15.132:0/1993075313 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f8c94003cb0 con 0x7f8c9c1010f0 2024-04-23T18:46:06.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.131+0000 7f8c83fff640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8c840a3dc0 0x7f8c840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.132+0000 7f8c9a59c640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8c840a3dc0 0x7f8c840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.132+0000 7f8c83fff640 1 -- 172.21.15.132:0/1993075313 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f8c940f10f0 con 0x7f8c9c1010f0 2024-04-23T18:46:06.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.132+0000 7f8c9a59c640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8c840a3dc0 0x7f8c840a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c06a150 tx=0x7f8c900075b0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:06.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.136+0000 7f8c83fff640 1 -- 172.21.15.132:0/1993075313 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c940bd100 con 0x7f8c9c1010f0 2024-04-23T18:46:06.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.335+0000 7f8ca1074640 1 -- 172.21.15.132:0/1993075313 --> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f8c9c109410 con 0x7f8c840a3dc0 2024-04-23T18:46:06.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.342+0000 7f8c83fff640 1 -- 172.21.15.132:0/1993075313 <== mgr.15366 v2:172.21.15.132:6800/3428544997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f8c90007b10 con 0x7f8c840a3dc0 2024-04-23T18:46:06.333 INFO:teuthology.orchestra.run.smithi132.stdout:/ 1m 6m3h 2024-04-23T18:46:06.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.344+0000 7f8ca1074640 1 -- 172.21.15.132:0/1993075313 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8c840a3dc0 msgr2=0x7f8c840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.344+0000 7f8ca1074640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8c840a3dc0 0x7f8c840a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c06a150 tx=0x7f8c900075b0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.344+0000 7f8ca1074640 1 -- 172.21.15.132:0/1993075313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9c1010f0 msgr2=0x7f8c9c1994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.344+0000 7f8ca1074640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9c1010f0 0x7f8c9c1994a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8c94006010 tx=0x7f8c940042f0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.344+0000 7f8ca1074640 1 -- 172.21.15.132:0/1993075313 shutdown_connections 2024-04-23T18:46:06.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.344+0000 7f8ca1074640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8c840a3dc0 0x7f8c840a6280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.344+0000 7f8ca1074640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c9c199f70 0x7f8c9c1a4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.344+0000 7f8ca1074640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c106170 0x7f8c9c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.344+0000 7f8ca1074640 1 --2- 172.21.15.132:0/1993075313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9c1010f0 0x7f8c9c1994a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.344+0000 7f8ca1074640 1 -- 172.21.15.132:0/1993075313 >> 172.21.15.132:0/1993075313 conn(0x7f8c9c077fd0 msgr2=0x7f8c9c0feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:06.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.344+0000 7f8ca1074640 1 -- 172.21.15.132:0/1993075313 shutdown_connections 2024-04-23T18:46:06.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.344+0000 7f8ca1074640 1 -- 172.21.15.132:0/1993075313 wait complete. 2024-04-23T18:46:06.347 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-23T18:46:06.361 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:46:06.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.468+0000 7f9e3a435640 1 -- 172.21.15.132:0/2015802363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e34102240 msgr2=0x7f9e34102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.468+0000 7f9e3a435640 1 --2- 172.21.15.132:0/2015802363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e34102240 0x7f9e34102620 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009900 tx=0x7f9e2802a410 comp rx=0 tx=0).stop 2024-04-23T18:46:06.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.468+0000 7f9e3a435640 1 -- 172.21.15.132:0/2015802363 shutdown_connections 2024-04-23T18:46:06.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.468+0000 7f9e3a435640 1 --2- 172.21.15.132:0/2015802363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e34102bf0 0x7f9e34106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.468+0000 7f9e3a435640 1 --2- 172.21.15.132:0/2015802363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e34107280 0x7f9e3410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.468+0000 7f9e3a435640 1 --2- 172.21.15.132:0/2015802363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e34102240 0x7f9e34102620 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.468+0000 7f9e3a435640 1 -- 172.21.15.132:0/2015802363 >> 172.21.15.132:0/2015802363 conn(0x7f9e340790e0 msgr2=0x7f9e340794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:06.462 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.468+0000 7f9e3a435640 1 -- 172.21.15.132:0/2015802363 shutdown_connections 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.468+0000 7f9e3a435640 1 -- 172.21.15.132:0/2015802363 wait complete. 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.468+0000 7f9e3a435640 1 Processor -- start 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.468+0000 7f9e3a435640 1 -- start start 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e3a435640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e34102240 0x7f9e3419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e3a435640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e34107280 0x7f9e3419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e3a435640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e341985e0 0x7f9e34198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e3a435640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9e34112280 con 0x7f9e341985e0 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e3a435640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e34112100 con 0x7f9e34102240 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e3a435640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9e34112400 con 0x7f9e34107280 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e389d2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e341985e0 0x7f9e34198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e389d2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e341985e0 0x7f9e34198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:47152/0 (socket says 172.21.15.132:47152) 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e389d2640 1 -- 172.21.15.132:0/853946483 learned_addr learned my addr 172.21.15.132:0/853946483 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:06.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e337fe640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e34107280 0x7f9e3419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e389d2640 1 -- 172.21.15.132:0/853946483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e34107280 msgr2=0x7f9e3419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e389d2640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e34107280 0x7f9e3419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e389d2640 1 -- 172.21.15.132:0/853946483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e34102240 msgr2=0x7f9e3419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e389d2640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e34102240 0x7f9e3419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e389d2640 1 -- 172.21.15.132:0/853946483 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e34199100 con 0x7f9e341985e0 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e337fe640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e34107280 0x7f9e3419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e389d2640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e341985e0 0x7f9e34198a90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00b970 tx=0x7f9e2c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e317fa640 1 -- 172.21.15.132:0/853946483 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c019070 con 0x7f9e341985e0 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e317fa640 1 -- 172.21.15.132:0/853946483 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e2c014440 con 0x7f9e341985e0 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e3a435640 1 -- 172.21.15.132:0/853946483 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e341993f0 con 0x7f9e341985e0 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e317fa640 1 -- 172.21.15.132:0/853946483 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c0075d0 con 0x7f9e341985e0 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.469+0000 7f9e3a435640 1 -- 172.21.15.132:0/853946483 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e341a5640 con 0x7f9e341985e0 2024-04-23T18:46:06.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.470+0000 7f9e3a435640 1 -- 172.21.15.132:0/853946483 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e340691c0 con 0x7f9e341985e0 2024-04-23T18:46:06.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.471+0000 7f9e317fa640 1 -- 172.21.15.132:0/853946483 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f9e2c01c070 con 0x7f9e341985e0 2024-04-23T18:46:06.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.472+0000 7f9e317fa640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9e1c0a3da0 0x7f9e1c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.472+0000 7f9e33fff640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9e1c0a3da0 0x7f9e1c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.472+0000 7f9e317fa640 1 -- 172.21.15.132:0/853946483 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f9e2c0d0230 con 0x7f9e341985e0 2024-04-23T18:46:06.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.473+0000 7f9e33fff640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9e1c0a3da0 0x7f9e1c0a6260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9e280096c0 tx=0x7f9e28031400 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:06.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.476+0000 7f9e317fa640 1 -- 172.21.15.132:0/853946483 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e2c09c9a0 con 0x7f9e341985e0 2024-04-23T18:46:06.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.655+0000 7f9e3a435640 1 -- 172.21.15.132:0/853946483 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e34199b10 con 0x7f9e341985e0 2024-04-23T18:46:06.647 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.656+0000 7f9e317fa640 1 -- 172.21.15.132:0/853946483 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23318 (secure 0 0 0) 0x7f9e2c09c2a0 con 0x7f9e341985e0 2024-04-23T18:46:06.648 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:46:06.648 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":97,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:45:15.658115+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":13,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/2124409853":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/3095429281":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/899407689":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1857586348":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":4,"length":1},{"begin":13,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:46:06.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.658+0000 7f9e3a435640 1 -- 172.21.15.132:0/853946483 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9e1c0a3da0 msgr2=0x7f9e1c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.658+0000 7f9e3a435640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9e1c0a3da0 0x7f9e1c0a6260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9e280096c0 tx=0x7f9e28031400 comp rx=0 tx=0).stop 2024-04-23T18:46:06.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.658+0000 7f9e3a435640 1 -- 172.21.15.132:0/853946483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e341985e0 msgr2=0x7f9e34198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.658+0000 7f9e3a435640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e341985e0 0x7f9e34198a90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00b970 tx=0x7f9e2c00be30 comp rx=0 tx=0).stop 2024-04-23T18:46:06.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.658+0000 7f9e3a435640 1 -- 172.21.15.132:0/853946483 shutdown_connections 2024-04-23T18:46:06.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.658+0000 7f9e3a435640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9e1c0a3da0 0x7f9e1c0a6260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.658+0000 7f9e3a435640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e34107280 0x7f9e3419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.658+0000 7f9e3a435640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e341985e0 0x7f9e34198a90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.658+0000 7f9e3a435640 1 --2- 172.21.15.132:0/853946483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e34102240 0x7f9e3419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.658+0000 7f9e3a435640 1 -- 172.21.15.132:0/853946483 >> 172.21.15.132:0/853946483 conn(0x7f9e340790e0 msgr2=0x7f9e34100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:06.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.658+0000 7f9e3a435640 1 -- 172.21.15.132:0/853946483 shutdown_connections 2024-04-23T18:46:06.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.658+0000 7f9e3a435640 1 -- 172.21.15.132:0/853946483 wait complete. 2024-04-23T18:46:06.660 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:29:20.132451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:33:58.407521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:33:59.518206+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:46:06.661 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:46:06.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.771+0000 7f8bb5f08640 1 -- 172.21.15.132:0/580588874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bb006af30 msgr2=0x7f8bb010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.771+0000 7f8bb5f08640 1 --2- 172.21.15.132:0/580588874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bb006af30 0x7f8bb010ee60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009cd0 tx=0x7f8ba802a8e0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.771+0000 7f8bb5f08640 1 -- 172.21.15.132:0/580588874 shutdown_connections 2024-04-23T18:46:06.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.771+0000 7f8bb5f08640 1 --2- 172.21.15.132:0/580588874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bb006a570 0x7f8bb006a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.771+0000 7f8bb5f08640 1 --2- 172.21.15.132:0/580588874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bb006af30 0x7f8bb010ee60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.771+0000 7f8bb5f08640 1 --2- 172.21.15.132:0/580588874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bb0106f90 0x7f8bb0107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.771+0000 7f8bb5f08640 1 -- 172.21.15.132:0/580588874 >> 172.21.15.132:0/580588874 conn(0x7f8bb0077e30 msgr2=0x7f8bb0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:06.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.771+0000 7f8bb5f08640 1 -- 172.21.15.132:0/580588874 shutdown_connections 2024-04-23T18:46:06.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.771+0000 7f8bb5f08640 1 -- 172.21.15.132:0/580588874 wait complete. 2024-04-23T18:46:06.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.771+0000 7f8bb5f08640 1 Processor -- start 2024-04-23T18:46:06.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.771+0000 7f8bb5f08640 1 -- start start 2024-04-23T18:46:06.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8bb5f08640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bb006a570 0x7f8bb0198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8bb5f08640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bb006af30 0x7f8bb0198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8bb5f08640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bb0106f90 0x7f8bb019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8bb5f08640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0112260 con 0x7f8bb0106f90 2024-04-23T18:46:06.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8bb5f08640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01120e0 con 0x7f8bb006a570 2024-04-23T18:46:06.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8bb5f08640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01123e0 con 0x7f8bb006af30 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8baffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bb0106f90 0x7f8bb019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8baffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bb0106f90 0x7f8bb019cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:47168/0 (socket says 172.21.15.132:47168) 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8baffff640 1 -- 172.21.15.132:0/1175205959 learned_addr learned my addr 172.21.15.132:0/1175205959 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8baeffd640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bb006af30 0x7f8bb0198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8baffff640 1 -- 172.21.15.132:0/1175205959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bb006af30 msgr2=0x7f8bb0198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8baffff640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bb006af30 0x7f8bb0198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8baffff640 1 -- 172.21.15.132:0/1175205959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bb006a570 msgr2=0x7f8bb0198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8baffff640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bb006a570 0x7f8bb0198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8baffff640 1 -- 172.21.15.132:0/1175205959 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb019d670 con 0x7f8bb0106f90 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8baeffd640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bb006af30 0x7f8bb0198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8baffff640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bb0106f90 0x7f8bb019cf40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8005e30 tx=0x7f8ba8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8bacff9640 1 -- 172.21.15.132:0/1175205959 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8004240 con 0x7f8bb0106f90 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8bacff9640 1 -- 172.21.15.132:0/1175205959 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ba8031ce0 con 0x7f8bb0106f90 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8bacff9640 1 -- 172.21.15.132:0/1175205959 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8046480 con 0x7f8bb0106f90 2024-04-23T18:46:06.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8bb5f08640 1 -- 172.21.15.132:0/1175205959 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb019d900 con 0x7f8bb0106f90 2024-04-23T18:46:06.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.772+0000 7f8bb5f08640 1 -- 172.21.15.132:0/1175205959 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb01a51f0 con 0x7f8bb0106f90 2024-04-23T18:46:06.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.774+0000 7f8bb5f08640 1 -- 172.21.15.132:0/1175205959 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb00691c0 con 0x7f8bb0106f90 2024-04-23T18:46:06.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.775+0000 7f8bacff9640 1 -- 172.21.15.132:0/1175205959 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f8ba80306a0 con 0x7f8bb0106f90 2024-04-23T18:46:06.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.775+0000 7f8bacff9640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8b980a3d20 0x7f8b980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.776+0000 7f8baf7fe640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8b980a3d20 0x7f8b980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.776+0000 7f8bacff9640 1 -- 172.21.15.132:0/1175205959 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f8ba802e030 con 0x7f8bb0106f90 2024-04-23T18:46:06.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.776+0000 7f8baf7fe640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8b980a3d20 0x7f8b980a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000a990 tx=0x7f8ba0005c70 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:06.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.779+0000 7f8bacff9640 1 -- 172.21.15.132:0/1175205959 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ba804b060 con 0x7f8bb0106f90 2024-04-23T18:46:06.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.958+0000 7f8bb5f08640 1 -- 172.21.15.132:0/1175205959 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8bb010a520 con 0x7f8bb0106f90 2024-04-23T18:46:06.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.959+0000 7f8bacff9640 1 -- 172.21.15.132:0/1175205959 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f8ba80be490 con 0x7f8bb0106f90 2024-04-23T18:46:06.950 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-04-23T18:46:06.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.961+0000 7f8bb5f08640 1 -- 172.21.15.132:0/1175205959 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8b980a3d20 msgr2=0x7f8b980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.961+0000 7f8bb5f08640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8b980a3d20 0x7f8b980a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000a990 tx=0x7f8ba0005c70 comp rx=0 tx=0).stop 2024-04-23T18:46:06.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.961+0000 7f8bb5f08640 1 -- 172.21.15.132:0/1175205959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bb0106f90 msgr2=0x7f8bb019cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:06.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.961+0000 7f8bb5f08640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bb0106f90 0x7f8bb019cf40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8005e30 tx=0x7f8ba8003ba0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.961+0000 7f8bb5f08640 1 -- 172.21.15.132:0/1175205959 shutdown_connections 2024-04-23T18:46:06.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.961+0000 7f8bb5f08640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8b980a3d20 0x7f8b980a61e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.961+0000 7f8bb5f08640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bb006af30 0x7f8bb0198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.961+0000 7f8bb5f08640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bb0106f90 0x7f8bb019cf40 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.961+0000 7f8bb5f08640 1 --2- 172.21.15.132:0/1175205959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bb006a570 0x7f8bb0198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.961+0000 7f8bb5f08640 1 -- 172.21.15.132:0/1175205959 >> 172.21.15.132:0/1175205959 conn(0x7f8bb0077e30 msgr2=0x7f8bb00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:06.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.961+0000 7f8bb5f08640 1 -- 172.21.15.132:0/1175205959 shutdown_connections 2024-04-23T18:46:06.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:06.961+0000 7f8bb5f08640 1 -- 172.21.15.132:0/1175205959 wait complete. 2024-04-23T18:46:06.962 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:06 smithi132 ceph-mon[29178]: pgmap v310: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:46:06.962 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:06 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/853946483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:46:06.963 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:46:06.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:06 smithi157 ceph-mon[32580]: pgmap v310: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:46:06.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:06 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/853946483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:46:07.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:06 smithi148 ceph-mon[32311]: pgmap v310: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:46:07.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:06 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/853946483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:46:07.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.073+0000 7f544eee1640 1 -- 172.21.15.132:0/1531356632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5448102170 msgr2=0x7f5448102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.073+0000 7f544eee1640 1 --2- 172.21.15.132:0/1531356632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5448102170 0x7f5448102550 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f543c009900 tx=0x7f543c02a410 comp rx=0 tx=0).stop 2024-04-23T18:46:07.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.073+0000 7f544eee1640 1 -- 172.21.15.132:0/1531356632 shutdown_connections 2024-04-23T18:46:07.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.073+0000 7f544eee1640 1 --2- 172.21.15.132:0/1531356632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5448102b20 0x7f5448106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.073+0000 7f544eee1640 1 --2- 172.21.15.132:0/1531356632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5448107240 0x7f544810ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.073+0000 7f544eee1640 1 --2- 172.21.15.132:0/1531356632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5448102170 0x7f5448102550 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.073+0000 7f544eee1640 1 -- 172.21.15.132:0/1531356632 >> 172.21.15.132:0/1531356632 conn(0x7f54480793a0 msgr2=0x7f54480797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.073+0000 7f544eee1640 1 -- 172.21.15.132:0/1531356632 shutdown_connections 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.073+0000 7f544eee1640 1 -- 172.21.15.132:0/1531356632 wait complete. 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544eee1640 1 Processor -- start 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544eee1640 1 -- start start 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544eee1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5448102170 0x7f5448198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544eee1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5448107240 0x7f5448198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544eee1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544819cfe0 0x7f544819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544eee1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f54481121a0 con 0x7f544819cfe0 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544eee1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5448112020 con 0x7f5448102170 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544eee1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5448112320 con 0x7f5448107240 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544d47e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544819cfe0 0x7f544819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544d47e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544819cfe0 0x7f544819d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:47188/0 (socket says 172.21.15.132:47188) 2024-04-23T18:46:07.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544d47e640 1 -- 172.21.15.132:0/2160803175 learned_addr learned my addr 172.21.15.132:0/2160803175 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:07.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544cc7d640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5448102170 0x7f5448198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f5447fff640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5448107240 0x7f5448198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544d47e640 1 -- 172.21.15.132:0/2160803175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5448107240 msgr2=0x7f5448198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544d47e640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5448107240 0x7f5448198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544d47e640 1 -- 172.21.15.132:0/2160803175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5448102170 msgr2=0x7f5448198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544d47e640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5448102170 0x7f5448198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544d47e640 1 -- 172.21.15.132:0/2160803175 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544819dad0 con 0x7f544819cfe0 2024-04-23T18:46:07.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544cc7d640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5448102170 0x7f5448198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:07.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.074+0000 7f544d47e640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544819cfe0 0x7f544819d490 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5440009fd0 tx=0x7f544000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:07.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.075+0000 7f5445ffb640 1 -- 172.21.15.132:0/2160803175 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440017070 con 0x7f544819cfe0 2024-04-23T18:46:07.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.075+0000 7f5445ffb640 1 -- 172.21.15.132:0/2160803175 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5440003e20 con 0x7f544819cfe0 2024-04-23T18:46:07.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.075+0000 7f544eee1640 1 -- 172.21.15.132:0/2160803175 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544819ddc0 con 0x7f544819cfe0 2024-04-23T18:46:07.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.075+0000 7f5445ffb640 1 -- 172.21.15.132:0/2160803175 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440010460 con 0x7f544819cfe0 2024-04-23T18:46:07.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.075+0000 7f544eee1640 1 -- 172.21.15.132:0/2160803175 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54481a55b0 con 0x7f544819cfe0 2024-04-23T18:46:07.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.076+0000 7f544eee1640 1 -- 172.21.15.132:0/2160803175 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54480691c0 con 0x7f544819cfe0 2024-04-23T18:46:07.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.077+0000 7f5445ffb640 1 -- 172.21.15.132:0/2160803175 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f5440003970 con 0x7f544819cfe0 2024-04-23T18:46:07.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.077+0000 7f5445ffb640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f54380a3da0 0x7f54380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.078+0000 7f544cc7d640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f54380a3da0 0x7f54380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.078+0000 7f5445ffb640 1 -- 172.21.15.132:0/2160803175 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f54400d05d0 con 0x7f544819cfe0 2024-04-23T18:46:07.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.078+0000 7f544cc7d640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f54380a3da0 0x7f54380a6260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f543c031440 tx=0x7f543c005a20 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:07.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.082+0000 7f5445ffb640 1 -- 172.21.15.132:0/2160803175 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f544009cc30 con 0x7f544819cfe0 2024-04-23T18:46:07.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.258+0000 7f544eee1640 1 -- 172.21.15.132:0/2160803175 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f544810a4d0 con 0x7f544819cfe0 2024-04-23T18:46:07.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.258+0000 7f5445ffb640 1 -- 172.21.15.132:0/2160803175 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f544009c530 con 0x7f544819cfe0 2024-04-23T18:46:07.250 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-04-23T18:46:07.251 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.260+0000 7f544eee1640 1 -- 172.21.15.132:0/2160803175 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f54380a3da0 msgr2=0x7f54380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.251 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.260+0000 7f544eee1640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f54380a3da0 0x7f54380a6260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f543c031440 tx=0x7f543c005a20 comp rx=0 tx=0).stop 2024-04-23T18:46:07.251 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.260+0000 7f544eee1640 1 -- 172.21.15.132:0/2160803175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544819cfe0 msgr2=0x7f544819d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.260+0000 7f544eee1640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544819cfe0 0x7f544819d490 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5440009fd0 tx=0x7f544000b260 comp rx=0 tx=0).stop 2024-04-23T18:46:07.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.260+0000 7f544eee1640 1 -- 172.21.15.132:0/2160803175 shutdown_connections 2024-04-23T18:46:07.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.260+0000 7f544eee1640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f54380a3da0 0x7f54380a6260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.260+0000 7f544eee1640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5448107240 0x7f5448198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.260+0000 7f544eee1640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544819cfe0 0x7f544819d490 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.260+0000 7f544eee1640 1 --2- 172.21.15.132:0/2160803175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5448102170 0x7f5448198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.260+0000 7f544eee1640 1 -- 172.21.15.132:0/2160803175 >> 172.21.15.132:0/2160803175 conn(0x7f54480793a0 msgr2=0x7f54480fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:07.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.260+0000 7f544eee1640 1 -- 172.21.15.132:0/2160803175 shutdown_connections 2024-04-23T18:46:07.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.260+0000 7f544eee1640 1 -- 172.21.15.132:0/2160803175 wait complete. 2024-04-23T18:46:07.262 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:46:07.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.372+0000 7fd2af16f640 1 -- 172.21.15.132:0/3987744422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8101a40 msgr2=0x7fd2a8105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.372+0000 7fd2af16f640 1 --2- 172.21.15.132:0/3987744422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8101a40 0x7fd2a8105a80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009900 tx=0x7fd29c02a410 comp rx=0 tx=0).stop 2024-04-23T18:46:07.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 -- 172.21.15.132:0/3987744422 shutdown_connections 2024-04-23T18:46:07.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 --2- 172.21.15.132:0/3987744422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a8106160 0x7fd2a810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 --2- 172.21.15.132:0/3987744422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8101a40 0x7fd2a8105a80 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 --2- 172.21.15.132:0/3987744422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2a8101090 0x7fd2a8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 -- 172.21.15.132:0/3987744422 >> 172.21.15.132:0/3987744422 conn(0x7fd2a8077f50 msgr2=0x7fd2a8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:07.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 -- 172.21.15.132:0/3987744422 shutdown_connections 2024-04-23T18:46:07.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 -- 172.21.15.132:0/3987744422 wait complete. 2024-04-23T18:46:07.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 Processor -- start 2024-04-23T18:46:07.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 -- start start 2024-04-23T18:46:07.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8101090 0x7fd2a8197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a8106160 0x7fd2a8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2a819bf20 0x7fd2a819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81110a0 con 0x7fd2a8101090 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8110f20 con 0x7fd2a819bf20 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2af16f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8111220 con 0x7fd2a8106160 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2acf0b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8101090 0x7fd2a8197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2acf0b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8101090 0x7fd2a8197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:47206/0 (socket says 172.21.15.132:47206) 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2acf0b640 1 -- 172.21.15.132:0/3328472058 learned_addr learned my addr 172.21.15.132:0/3328472058 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.373+0000 7fd2a7fff640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a8106160 0x7fd2a8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2ad70c640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2a819bf20 0x7fd2a819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2acf0b640 1 -- 172.21.15.132:0/3328472058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a8106160 msgr2=0x7fd2a8197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2acf0b640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a8106160 0x7fd2a8197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2acf0b640 1 -- 172.21.15.132:0/3328472058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2a819bf20 msgr2=0x7fd2a819c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2acf0b640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2a819bf20 0x7fd2a819c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2acf0b640 1 -- 172.21.15.132:0/3328472058 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a819ca10 con 0x7fd2a8101090 2024-04-23T18:46:07.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2ad70c640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2a819bf20 0x7fd2a819c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:07.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2acf0b640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8101090 0x7fd2a8197660 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd29800d950 tx=0x7fd29800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:07.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2a5ffb640 1 -- 172.21.15.132:0/3328472058 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298017070 con 0x7fd2a8101090 2024-04-23T18:46:07.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2a5ffb640 1 -- 172.21.15.132:0/3328472058 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd298009c30 con 0x7fd2a8101090 2024-04-23T18:46:07.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2a5ffb640 1 -- 172.21.15.132:0/3328472058 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29800f620 con 0x7fd2a8101090 2024-04-23T18:46:07.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2af16f640 1 -- 172.21.15.132:0/3328472058 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a819cd00 con 0x7fd2a8101090 2024-04-23T18:46:07.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.374+0000 7fd2af16f640 1 -- 172.21.15.132:0/3328472058 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a81a4510 con 0x7fd2a8101090 2024-04-23T18:46:07.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.375+0000 7fd2af16f640 1 -- 172.21.15.132:0/3328472058 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a8068d70 con 0x7fd2a8101090 2024-04-23T18:46:07.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.376+0000 7fd2a5ffb640 1 -- 172.21.15.132:0/3328472058 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fd298010460 con 0x7fd2a8101090 2024-04-23T18:46:07.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.377+0000 7fd2a5ffb640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fd2900a3da0 0x7fd2900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.377+0000 7fd2a7fff640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fd2900a3da0 0x7fd2900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.377+0000 7fd2a5ffb640 1 -- 172.21.15.132:0/3328472058 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7fd2980d0440 con 0x7fd2a8101090 2024-04-23T18:46:07.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.377+0000 7fd2a7fff640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fd2900a3da0 0x7fd2900a6260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd29c000c00 tx=0x7fd29c031400 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:07.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.380+0000 7fd2a5ffb640 1 -- 172.21.15.132:0/3328472058 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd29809cac0 con 0x7fd2a8101090 2024-04-23T18:46:07.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.557+0000 7fd2af16f640 1 -- 172.21.15.132:0/3328472058 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd2a81093f0 con 0x7fd2a8101090 2024-04-23T18:46:07.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.558+0000 7fd2a5ffb640 1 -- 172.21.15.132:0/3328472058 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7fd298009de0 con 0x7fd2a8101090 2024-04-23T18:46:07.549 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-04-23T18:46:07.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.560+0000 7fd2af16f640 1 -- 172.21.15.132:0/3328472058 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fd2900a3da0 msgr2=0x7fd2900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.560+0000 7fd2af16f640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fd2900a3da0 0x7fd2900a6260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd29c000c00 tx=0x7fd29c031400 comp rx=0 tx=0).stop 2024-04-23T18:46:07.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.560+0000 7fd2af16f640 1 -- 172.21.15.132:0/3328472058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8101090 msgr2=0x7fd2a8197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.560+0000 7fd2af16f640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8101090 0x7fd2a8197660 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd29800d950 tx=0x7fd29800de10 comp rx=0 tx=0).stop 2024-04-23T18:46:07.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.560+0000 7fd2af16f640 1 -- 172.21.15.132:0/3328472058 shutdown_connections 2024-04-23T18:46:07.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.560+0000 7fd2af16f640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fd2900a3da0 0x7fd2900a6260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.560+0000 7fd2af16f640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a8106160 0x7fd2a8197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.560+0000 7fd2af16f640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2a819bf20 0x7fd2a819c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.560+0000 7fd2af16f640 1 --2- 172.21.15.132:0/3328472058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8101090 0x7fd2a8197660 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.560+0000 7fd2af16f640 1 -- 172.21.15.132:0/3328472058 >> 172.21.15.132:0/3328472058 conn(0x7fd2a8077f50 msgr2=0x7fd2a80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:07.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.560+0000 7fd2af16f640 1 -- 172.21.15.132:0/3328472058 shutdown_connections 2024-04-23T18:46:07.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.560+0000 7fd2af16f640 1 -- 172.21.15.132:0/3328472058 wait complete. 2024-04-23T18:46:07.562 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-23T18:46:07.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.669+0000 7f9b4e272640 1 -- 172.21.15.132:0/3116889668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b4810c520 msgr2=0x7f9b4810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.669+0000 7f9b4e272640 1 --2- 172.21.15.132:0/3116889668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b4810c520 0x7f9b4810e9c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009ca0 tx=0x7f9b4002a9e0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.669+0000 7f9b4e272640 1 -- 172.21.15.132:0/3116889668 shutdown_connections 2024-04-23T18:46:07.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.669+0000 7f9b4e272640 1 --2- 172.21.15.132:0/3116889668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b48103b00 0x7f9b4810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.669+0000 7f9b4e272640 1 --2- 172.21.15.132:0/3116889668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b4810c520 0x7f9b4810e9c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.669+0000 7f9b4e272640 1 --2- 172.21.15.132:0/3116889668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b481031e0 0x7f9b481035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.669+0000 7f9b4e272640 1 -- 172.21.15.132:0/3116889668 >> 172.21.15.132:0/3116889668 conn(0x7f9b48077f50 msgr2=0x7f9b48078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:07.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.670+0000 7f9b4e272640 1 -- 172.21.15.132:0/3116889668 shutdown_connections 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.670+0000 7f9b4e272640 1 -- 172.21.15.132:0/3116889668 wait complete. 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.670+0000 7f9b4e272640 1 Processor -- start 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.670+0000 7f9b4e272640 1 -- start start 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.670+0000 7f9b4e272640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b481031e0 0x7f9b48198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.670+0000 7f9b4e272640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b48103b00 0x7f9b48198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.670+0000 7f9b4e272640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b4810c520 0x7f9b4819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.670+0000 7f9b4e272640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48112210 con 0x7f9b48103b00 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.670+0000 7f9b4e272640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48112090 con 0x7f9b481031e0 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.670+0000 7f9b4e272640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48112390 con 0x7f9b4810c520 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b477fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b48103b00 0x7f9b48198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b477fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b48103b00 0x7f9b48198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:47238/0 (socket says 172.21.15.132:47238) 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b477fe640 1 -- 172.21.15.132:0/551979836 learned_addr learned my addr 172.21.15.132:0/551979836 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b4c80f640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b4810c520 0x7f9b4819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b47fff640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b481031e0 0x7f9b48198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b477fe640 1 -- 172.21.15.132:0/551979836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b4810c520 msgr2=0x7f9b4819cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b477fe640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b4810c520 0x7f9b4819cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b477fe640 1 -- 172.21.15.132:0/551979836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b481031e0 msgr2=0x7f9b48198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b477fe640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b481031e0 0x7f9b48198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b477fe640 1 -- 172.21.15.132:0/551979836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4819d700 con 0x7f9b48103b00 2024-04-23T18:46:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b4c80f640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b4810c520 0x7f9b4819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:46:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b477fe640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b48103b00 0x7f9b48198c50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00c890 tx=0x7f9b3c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b457fa640 1 -- 172.21.15.132:0/551979836 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c004220 con 0x7f9b48103b00 2024-04-23T18:46:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b457fa640 1 -- 172.21.15.132:0/551979836 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b3c0043c0 con 0x7f9b48103b00 2024-04-23T18:46:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b457fa640 1 -- 172.21.15.132:0/551979836 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c004a00 con 0x7f9b48103b00 2024-04-23T18:46:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b4e272640 1 -- 172.21.15.132:0/551979836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4819d9f0 con 0x7f9b48103b00 2024-04-23T18:46:07.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.671+0000 7f9b4e272640 1 -- 172.21.15.132:0/551979836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b481a5280 con 0x7f9b48103b00 2024-04-23T18:46:07.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.672+0000 7f9b4e272640 1 -- 172.21.15.132:0/551979836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b48068d70 con 0x7f9b48103b00 2024-04-23T18:46:07.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.674+0000 7f9b457fa640 1 -- 172.21.15.132:0/551979836 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f9b3c0079b0 con 0x7f9b48103b00 2024-04-23T18:46:07.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.674+0000 7f9b457fa640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9b300a3da0 0x7f9b300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.674+0000 7f9b457fa640 1 -- 172.21.15.132:0/551979836 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f9b3c0d0990 con 0x7f9b48103b00 2024-04-23T18:46:07.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.674+0000 7f9b47fff640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9b300a3da0 0x7f9b300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.675+0000 7f9b47fff640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9b300a3da0 0x7f9b300a6260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800a810 tx=0x7f9b38008040 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:07.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.680+0000 7f9b457fa640 1 -- 172.21.15.132:0/551979836 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b3c014030 con 0x7f9b48103b00 2024-04-23T18:46:07.860 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:07 smithi132 ceph-mon[29178]: from='client.15912 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:46:07.860 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:07 smithi132 ceph-mon[29178]: from='client.15918 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:46:07.860 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:07 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1175205959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:46:07.860 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:07 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2160803175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:46:07.861 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:07 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3328472058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:46:07.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.907+0000 7f9b4e272640 1 -- 172.21.15.132:0/551979836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f9b48199a60 con 0x7f9b48103b00 2024-04-23T18:46:07.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.914+0000 7f9b457fa640 1 -- 172.21.15.132:0/551979836 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746071 (secure 0 0 0) 0x7f9b48199a60 con 0x7f9b48103b00 2024-04-23T18:46:07.915 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: "epoch": 26, 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: "active_gid": 15366, 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: "active_name": "x", 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: "active_addrs": { 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:6800", 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 3428544997 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:07.916 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:6801", 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 3428544997 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: "active_addr": "172.21.15.132:6801/3428544997", 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: "active_change": "2024-04-23T18:36:02.717490+0000", 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: "active_mgr_features": 4540701547738038271, 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: "available": true, 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: "standbys": [ 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: "gid": 15360, 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "y", 2024-04-23T18:46:07.917 INFO:teuthology.orchestra.run.smithi132.stdout: "mgr_features": 4540701547738038271, 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "available_modules": [ 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "alerts", 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.918 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:07.919 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:07.920 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:07.921 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.922 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_destination": { 2024-04-23T18:46:07.923 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_destination", 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Email address to send alerts to", 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_from_name": { 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_from_name", 2024-04-23T18:46:07.924 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "Ceph", 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Email From: name", 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_host": { 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_host", 2024-04-23T18:46:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP server", 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_password": { 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_password", 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.926 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Password to authenticate with", 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_port": { 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_port", 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.927 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "465", 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP port", 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_sender": { 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_sender", 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.928 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP envelope sender", 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_ssl": { 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_ssl", 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:07.929 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_user": { 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_user", 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:07.930 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "User to authenticate as", 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "balancer", 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:07.931 INFO:teuthology.orchestra.run.smithi132.stdout: "active": { 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "active", 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.932 INFO:teuthology.orchestra.run.smithi132.stdout: "begin_time": { 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "begin_time", 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0000", 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "begin_weekday": { 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "begin_weekday", 2024-04-23T18:46:07.933 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:07.934 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.934 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.934 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:07.934 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0", 2024-04-23T18:46:07.934 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "6", 2024-04-23T18:46:07.934 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.934 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T18:46:07.934 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_max_iterations": { 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_max_iterations", 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "25", 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "250", 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.935 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_metrics": { 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_metrics", 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.936 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_step": { 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_step", 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0.5", 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0.001", 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "0.999", 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "aggressiveness of optimization", 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.937 INFO:teuthology.orchestra.run.smithi132.stdout: "end_time": { 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "end_time", 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "2359", 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "end_weekday": { 2024-04-23T18:46:07.938 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "end_weekday", 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0", 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "6", 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:07.939 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:07.940 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.941 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.942 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:07.943 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "min_score": { 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "min_score", 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.944 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "mode": { 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mode", 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "upmap", 2024-04-23T18:46:07.945 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: "crush-compat", 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: "none", 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: "read", 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap", 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap-read" 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Balancer mode", 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.946 INFO:teuthology.orchestra.run.smithi132.stdout: "pool_ids": { 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pool_ids", 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:07.947 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap_max_deviation": { 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "upmap_max_deviation", 2024-04-23T18:46:07.948 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap_max_optimizations": { 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "upmap_max_optimizations", 2024-04-23T18:46:07.949 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:07.950 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cephadm", 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_down_multiplier": { 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_down_multiplier", 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3.0", 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T18:46:07.951 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_refresh_rate": { 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_refresh_rate", 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "20", 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T18:46:07.952 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_starting_port": { 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_starting_port", 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "4721", 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T18:46:07.953 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_ptrace": { 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_ptrace", 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T18:46:07.954 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "autotune_interval": { 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "autotune_interval", 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T18:46:07.955 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: "autotune_memory_target_ratio": { 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0.7", 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.956 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm_log_destination": { 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cephadm_log_destination", 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "file", 2024-04-23T18:46:07.957 INFO:teuthology.orchestra.run.smithi132.stdout: "file,syslog", 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: "syslog" 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: "cgroups_split": { 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cgroups_split", 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:07.958 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "config_checks_enabled": { 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_checks_enabled", 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.959 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "config_dashboard": { 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_dashboard", 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.960 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_alertmanager": { 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_alertmanager", 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:07.961 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_base": { 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_base", 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Container image name, without the tag", 2024-04-23T18:46:07.962 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_elasticsearch": { 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_elasticsearch", 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "elasticsearch container image", 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.963 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_grafana": { 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_grafana", 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.964 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_haproxy": { 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_haproxy", 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "HAproxy container image", 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.965 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_agent": { 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_agent", 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger agent container image", 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.966 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_collector": { 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_collector", 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger collector container image", 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.967 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_query": { 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_query", 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger query container image", 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.968 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_keepalived": { 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_keepalived", 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Keepalived container image", 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.969 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_loki": { 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_loki", 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Loki container image", 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.970 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_node_exporter": { 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_node_exporter", 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.971 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_nvmeof": { 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_nvmeof", 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Nvme-of container image", 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_prometheus": { 2024-04-23T18:46:07.972 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_prometheus", 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_promtail": { 2024-04-23T18:46:07.973 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_promtail", 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Promtail container image", 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_samba": { 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_samba", 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.974 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Samba/SMB container image", 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_snmp_gateway": { 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_snmp_gateway", 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.975 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SNMP Gateway container image", 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "container_init": { 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_init", 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.976 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: "daemon_cache_timeout": { 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "daemon_cache_timeout", 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.977 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: "default_cephadm_command_timeout": { 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:07.978 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "900", 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "default_registry": { 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "default_registry", 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.979 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io", 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "device_cache_timeout": { 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_cache_timeout", 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.980 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1800", 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache device inventory", 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "device_enhanced_scan": { 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_enhanced_scan", 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.981 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "facts_cache_timeout": { 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "facts_cache_timeout", 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.982 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache host facts data", 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "grafana_dashboards_path": { 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "grafana_dashboards_path", 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T18:46:07.983 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: "host_check_interval": { 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "host_check_interval", 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.984 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to perform a host check", 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "hw_monitoring": { 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "hw_monitoring", 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.985 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "inventory_list_all": { 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "inventory_list_all", 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.986 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.987 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.988 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "log_refresh_metadata": { 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_refresh_metadata", 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.989 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:07.991 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.992 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.993 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "*", 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.994 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "max_count_per_host": { 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_count_per_host", 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.995 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "max number of daemons per service per host", 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "max_osd_draining_count": { 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_osd_draining_count", 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T18:46:07.996 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "migration_current": { 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "migration_current", 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "internal - do not modify", 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.997 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "mode": { 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mode", 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "root", 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm-package", 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "root" 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:07.998 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "oob_default_addr": { 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "oob_default_addr", 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "169.254.1.1", 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:07.999 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "prometheus_alerts_path": { 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prometheus_alerts_path", 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.000 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_insecure": { 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_insecure", 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T18:46:08.001 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_password": { 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_password", 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.002 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_url": { 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_url", 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.003 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_username": { 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_username", 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.004 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "secure_monitoring_stack": { 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "secure_monitoring_stack", 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.005 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "service_discovery_port": { 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "service_discovery_port", 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8765", 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "cephadm service discovery port", 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.006 INFO:teuthology.orchestra.run.smithi132.stdout: "ssh_config_file": { 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssh_config_file", 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.007 INFO:teuthology.orchestra.run.smithi132.stdout: "use_agent": { 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "use_agent", 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "use_repo_digest": { 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "use_repo_digest", 2024-04-23T18:46:08.008 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_failed_host_check": { 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_failed_host_check", 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.009 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_stray_daemons": { 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_stray_daemons", 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.010 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_stray_hosts": { 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_stray_hosts", 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.011 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crash", 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.012 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.013 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.014 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.015 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.016 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "retain_interval": { 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "retain_interval", 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.017 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "31536000", 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_recent_interval": { 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_recent_interval", 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.018 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1209600", 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "dashboard", 2024-04-23T18:46:08.019 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.020 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.021 INFO:teuthology.orchestra.run.smithi132.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.022 INFO:teuthology.orchestra.run.smithi132.stdout: "AUDIT_API_ENABLED": { 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T18:46:08.023 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.024 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.025 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.026 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.027 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.028 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.029 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.030 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.031 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.032 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.033 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_URL": { 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_URL", 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.035 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.036 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.037 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.038 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T18:46:08.039 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T18:46:08.040 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "[]", 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.041 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "{}", 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.042 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.043 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.044 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.045 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.046 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.047 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.049 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.050 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.051 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.052 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.053 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8", 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.054 INFO:teuthology.orchestra.run.smithi132.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "45", 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T18:46:08.055 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T18:46:08.056 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.057 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.058 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.059 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.060 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.061 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.062 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "cross_origin_url": { 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cross_origin_url", 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.063 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "crt_file": { 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crt_file", 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.064 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "debug": { 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "debug", 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.065 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable/disable debug options", 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "jwt_token_ttl": { 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "jwt_token_ttl", 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "28800", 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.066 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "key_file": { 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "key_file", 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.067 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.068 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.069 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.070 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.071 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.072 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "motd": { 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "motd", 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "The message of the day", 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "redirect_resolve_ip_addr": { 2024-04-23T18:46:08.073 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-04-23T18:46:08.074 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "::", 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-04-23T18:46:08.075 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8080", 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl": { 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl", 2024-04-23T18:46:08.076 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl_server_port": { 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl_server_port", 2024-04-23T18:46:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8443", 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_behaviour": { 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_behaviour", 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.078 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "redirect", 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: "redirect" 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.079 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_error_status_code": { 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_error_status_code", 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "500", 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "400", 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "599", 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "url_prefix": { 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "url_prefix", 2024-04-23T18:46:08.080 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.081 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "devicehealth", 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "enable_monitoring": { 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enable_monitoring", 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "monitor device health metrics", 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.082 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.083 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.084 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.085 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.086 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "mark_out_threshold": { 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mark_out_threshold", 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.087 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "2419200", 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "pool_name": { 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pool_name", 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.088 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "device_health_metrics", 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "retention_period": { 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "retention_period", 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.089 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15552000", 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long to retain device health metrics", 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "scrape_frequency": { 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "scrape_frequency", 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.090 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "86400", 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "self_heal": { 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "self_heal", 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.091 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:08.092 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_threshold": { 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_threshold", 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "7257600", 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.093 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "diskprediction_local", 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.094 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.095 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.096 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.097 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.098 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "predict_interval": { 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "predict_interval", 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "86400", 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.099 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "predictor_model": { 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "predictor_model", 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "prophetstor", 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.100 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.101 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "influx", 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": false, 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "influxdb python module not found", 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: "batch_size": { 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "batch_size", 2024-04-23T18:46:08.102 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5000", 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "database": { 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "database", 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.103 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "ceph", 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "hostname": { 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "hostname", 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.104 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB server hostname", 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "30", 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "5", 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.106 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.107 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.108 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.109 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.110 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "password": { 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "password", 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "password of InfluxDB server user", 2024-04-23T18:46:08.111 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "port": { 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "port", 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8086", 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB server port", 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.112 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl": { 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl", 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "false", 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.113 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "threads": { 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "threads", 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "32", 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.114 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "username": { 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "username", 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "username of InfluxDB server user", 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.115 INFO:teuthology.orchestra.run.smithi132.stdout: "verify_ssl": { 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "verify_ssl", 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "true", 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.116 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "insights", 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.117 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.118 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.119 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.120 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.121 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "iostat", 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.122 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.123 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.124 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.125 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.126 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "k8sevents", 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_event_retention_days": { 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ceph_event_retention_days", 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.127 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "7", 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "config_check_secs": { 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_check_secs", 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.128 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "10", 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.129 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.130 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.131 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.132 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.133 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "localpool", 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: "failure_domain": { 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "failure_domain", 2024-04-23T18:46:08.134 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "host", 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "failure domain for any created local pool", 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.135 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.136 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.137 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.138 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.139 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "min_size": { 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "min_size", 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.140 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default min_size for any created local pool", 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "num_rep": { 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "num_rep", 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3", 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.141 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default replica count for any created local pool", 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_num": { 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pg_num", 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "128", 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T18:46:08.142 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "prefix": { 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prefix", 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "name prefix for any created local pool", 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.143 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "subtree": { 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "subtree", 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "rack", 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.144 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mds_autoscaler", 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.145 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.146 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.147 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.148 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.149 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mirroring", 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.150 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.151 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.152 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.153 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.154 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "nfs", 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.155 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.156 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.157 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.158 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.159 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "orchestrator", 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: "fail_fs": { 2024-04-23T18:46:08.160 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "fail_fs", 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.161 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.162 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.163 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.164 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.165 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator": { 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "orchestrator", 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.166 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm", 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: "rook", 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: "test_orchestrator" 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Orchestrator backend", 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.167 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "osd_perf_query", 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.168 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.169 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.170 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.171 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.172 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "osd_support", 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.173 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.174 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.175 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.176 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.177 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pg_autoscaler", 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.178 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.179 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.180 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.181 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.182 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.183 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "threshold": { 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "threshold", 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3.0", 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1.0", 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "scaling threshold", 2024-04-23T18:46:08.184 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "progress", 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_pg_recovery_event": { 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_pg_recovery_event", 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.185 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "enabled": { 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enabled", 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.186 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.187 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.188 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.189 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.190 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.191 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "max_completed_events": { 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_completed_events", 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "50", 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.192 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "number of past completed events to remember", 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.193 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long the module is going to sleep", 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prometheus", 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: "cache": { 2024-04-23T18:46:08.194 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cache", 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "exclude_perf_counters": { 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "exclude_perf_counters", 2024-04-23T18:46:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.196 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.197 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.198 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.199 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.200 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_stats_pools": { 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_stats_pools", 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.201 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "300", 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.202 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "scrape_interval": { 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "scrape_interval", 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15.0", 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.203 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "::", 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.204 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "9283", 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.205 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "stale_cache_strategy": { 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "stale_cache_strategy", 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "log", 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.206 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_behaviour": { 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_behaviour", 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "default", 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.207 INFO:teuthology.orchestra.run.smithi132.stdout: "default", 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "error" 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_error_status_code": { 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_error_status_code", 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "500", 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "400", 2024-04-23T18:46:08.208 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "599", 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_support", 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.209 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.210 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.211 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.212 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.213 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "max_concurrent_snap_create": { 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_concurrent_snap_create", 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.214 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "mirror_snapshot_schedule": { 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.215 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.216 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.216 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.216 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.216 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.216 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.216 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.216 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.216 INFO:teuthology.orchestra.run.smithi132.stdout: "trash_purge_schedule": { 2024-04-23T18:46:08.216 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "trash_purge_schedule", 2024-04-23T18:46:08.216 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.216 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.216 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "restful", 2024-04-23T18:46:08.217 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "enable_auth": { 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enable_auth", 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.218 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "key_file": { 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "key_file", 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.219 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.220 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.221 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.222 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.223 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.224 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rgw", 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.226 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.227 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.228 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.229 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.230 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rook", 2024-04-23T18:46:08.231 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.232 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.233 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.234 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.235 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "storage_class": { 2024-04-23T18:46:08.236 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "storage_class", 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "local", 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "selftest", 2024-04-23T18:46:08.237 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.238 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.239 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.240 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.241 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "roption1": { 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "roption1", 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.242 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "roption2": { 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "roption2", 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "xyz", 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.243 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption1": { 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption1", 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.244 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption2": { 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption2", 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.245 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption3": { 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption3", 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.246 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption4": { 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption4", 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption5": { 2024-04-23T18:46:08.247 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption5", 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption6": { 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption6", 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.248 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption7": { 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption7", 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.249 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "42", 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "testkey": { 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testkey", 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "testlkey": { 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testlkey", 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.251 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "testnewline": { 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testnewline", 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.252 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snap_schedule", 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_m_granularity": { 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_m_granularity", 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.253 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: "dump_on_update": { 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "dump_on_update", 2024-04-23T18:46:08.254 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "dump database to debug log on update", 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.255 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.257 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.258 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.259 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.260 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "stats", 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.261 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.262 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.263 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.264 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.265 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "status", 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.266 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.267 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.268 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.270 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "telegraf", 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: "address": { 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "address", 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.271 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.272 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15", 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.273 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.275 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.276 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.277 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "telemetry", 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_basic": { 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_basic", 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.278 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_crash": { 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_crash", 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.281 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_device": { 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_device", 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.282 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.283 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.283 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.283 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.283 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T18:46:08.283 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.283 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.283 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.283 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.283 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_ident": { 2024-04-23T18:46:08.284 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_ident", 2024-04-23T18:46:08.284 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.284 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.284 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.284 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.284 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.284 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.284 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.285 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T18:46:08.285 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.285 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.285 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.285 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.285 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_perf": { 2024-04-23T18:46:08.285 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_perf", 2024-04-23T18:46:08.285 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.285 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.285 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.285 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.286 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.286 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.286 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.286 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T18:46:08.286 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.286 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.286 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.286 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.286 INFO:teuthology.orchestra.run.smithi132.stdout: "contact": { 2024-04-23T18:46:08.286 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "contact", 2024-04-23T18:46:08.286 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.287 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.287 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.287 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.287 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.287 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.287 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.287 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.287 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.287 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.287 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.287 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.288 INFO:teuthology.orchestra.run.smithi132.stdout: "description": { 2024-04-23T18:46:08.288 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "description", 2024-04-23T18:46:08.288 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.288 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.288 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.288 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.288 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.288 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.288 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.288 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.288 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.289 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.289 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.289 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.289 INFO:teuthology.orchestra.run.smithi132.stdout: "device_url": { 2024-04-23T18:46:08.289 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_url", 2024-04-23T18:46:08.289 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.289 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.289 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.289 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T18:46:08.289 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.289 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.290 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.290 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.290 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.290 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.290 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.290 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.290 INFO:teuthology.orchestra.run.smithi132.stdout: "enabled": { 2024-04-23T18:46:08.290 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enabled", 2024-04-23T18:46:08.290 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.290 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.290 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.291 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.291 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.291 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.291 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.291 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.291 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.291 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.291 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.291 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.291 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:08.291 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:08.291 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "24", 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "8", 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: "last_opt_revision": { 2024-04-23T18:46:08.292 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "last_opt_revision", 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1", 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.293 INFO:teuthology.orchestra.run.smithi132.stdout: "leaderboard": { 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "leaderboard", 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.294 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "leaderboard_description": { 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "leaderboard_description", 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.296 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.296 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.296 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.296 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.296 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.296 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.296 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.296 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.296 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.296 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.296 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.296 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.297 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.297 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.297 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.297 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.297 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.297 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.297 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.297 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.297 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.297 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.297 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.297 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.298 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.298 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.298 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.298 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.298 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.298 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.298 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.298 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.298 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.298 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.298 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.298 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.299 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.300 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.300 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.300 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.300 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.300 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.300 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.300 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.300 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.300 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.300 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.300 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "organization": { 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "organization", 2024-04-23T18:46:08.301 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: "proxy": { 2024-04-23T18:46:08.302 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "proxy", 2024-04-23T18:46:08.303 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.303 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.303 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.303 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.303 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.303 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.303 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.303 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.303 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.303 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.303 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.304 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.304 INFO:teuthology.orchestra.run.smithi132.stdout: "url": { 2024-04-23T18:46:08.304 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "url", 2024-04-23T18:46:08.304 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.304 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.304 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.304 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T18:46:08.304 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.304 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.304 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.304 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.304 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "test_orchestrator", 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.305 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.306 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.307 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.308 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.309 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.309 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.309 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.309 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.309 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.309 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.309 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.309 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.309 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.309 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.309 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.309 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.310 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.310 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.310 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.310 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.310 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.310 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.310 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.310 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.310 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.310 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.310 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "volumes", 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.311 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.312 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.315 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.316 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: "max_concurrent_clones": { 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_concurrent_clones", 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "4", 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.317 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.318 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T18:46:08.318 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.318 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.318 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.318 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.318 INFO:teuthology.orchestra.run.smithi132.stdout: "periodic_async_work": { 2024-04-23T18:46:08.318 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "periodic_async_work", 2024-04-23T18:46:08.318 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.318 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.318 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.318 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.318 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.319 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.319 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.319 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Periodically check for async work", 2024-04-23T18:46:08.319 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.319 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.319 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.319 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.319 INFO:teuthology.orchestra.run.smithi132.stdout: "snapshot_clone_delay": { 2024-04-23T18:46:08.319 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snapshot_clone_delay", 2024-04-23T18:46:08.319 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.319 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.319 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.320 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:08.320 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.320 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.320 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.320 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T18:46:08.320 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.320 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.320 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.320 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.320 INFO:teuthology.orchestra.run.smithi132.stdout: "snapshot_clone_no_wait": { 2024-04-23T18:46:08.320 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T18:46:08.320 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.321 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.321 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.321 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.321 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.321 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.321 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.321 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T18:46:08.321 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.321 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.321 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.321 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.321 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.322 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.322 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.322 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix", 2024-04-23T18:46:08.322 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.322 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.322 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.322 INFO:teuthology.orchestra.run.smithi132.stdout: "discovery_interval": { 2024-04-23T18:46:08.322 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "discovery_interval", 2024-04-23T18:46:08.322 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:08.322 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.322 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.322 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "100", 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: "identifier": { 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "identifier", 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.323 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.324 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.324 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.324 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.324 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.324 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.324 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.324 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.324 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.324 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.324 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:08.324 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:08.324 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.325 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.326 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.326 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.326 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.326 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.326 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.326 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.326 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.326 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.326 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.326 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.326 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.326 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.327 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.327 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.327 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.327 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.327 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.327 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.327 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.327 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.327 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.328 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.328 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.328 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.328 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.328 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.328 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.328 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.328 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.328 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.328 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.328 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.328 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.329 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.330 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.330 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.330 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.330 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.330 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.330 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.330 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.330 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.330 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.330 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.330 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.331 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.331 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.331 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.331 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.331 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.331 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.331 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.331 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.331 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.331 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.331 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_host": { 2024-04-23T18:46:08.331 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_host", 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_port": { 2024-04-23T18:46:08.332 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_port", 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10051", 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.333 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_sender": { 2024-04-23T18:46:08.334 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_sender", 2024-04-23T18:46:08.334 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.334 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.334 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.334 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T18:46:08.334 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.334 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.334 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.334 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.334 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.334 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.335 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.335 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.335 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.335 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.335 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-04-23T18:46:08.335 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.335 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.335 INFO:teuthology.orchestra.run.smithi132.stdout: "gid": 24862, 2024-04-23T18:46:08.335 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "z", 2024-04-23T18:46:08.335 INFO:teuthology.orchestra.run.smithi132.stdout: "mgr_features": 4540701547738038271, 2024-04-23T18:46:08.335 INFO:teuthology.orchestra.run.smithi132.stdout: "available_modules": [ 2024-04-23T18:46:08.335 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.336 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "alerts", 2024-04-23T18:46:08.336 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.336 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.336 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.336 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:08.336 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:08.336 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.336 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.336 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.336 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:08.336 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.336 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.337 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.337 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T18:46:08.337 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.337 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.337 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.337 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.337 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.337 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.337 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.337 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.337 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.337 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.338 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.338 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.338 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.338 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.338 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.338 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.338 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.338 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.338 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.338 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.338 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.338 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.339 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.339 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.339 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.339 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.339 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.339 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.339 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.339 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.339 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.339 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.339 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.339 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.340 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.340 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.340 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.340 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.340 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.340 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.340 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.340 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.340 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.340 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.340 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.340 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.341 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.341 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.341 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.341 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.341 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.341 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.341 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.341 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.341 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.341 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.341 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.341 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.342 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.342 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.342 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.342 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.342 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.342 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.342 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.342 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.342 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.342 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.342 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.342 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_destination": { 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_destination", 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.343 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Email address to send alerts to", 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_from_name": { 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_from_name", 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "Ceph", 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.344 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.345 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.345 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Email From: name", 2024-04-23T18:46:08.345 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.345 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.345 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.345 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.345 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_host": { 2024-04-23T18:46:08.345 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_host", 2024-04-23T18:46:08.345 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.345 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.345 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.345 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.346 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.346 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.346 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.346 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP server", 2024-04-23T18:46:08.346 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.346 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.346 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.346 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.346 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_password": { 2024-04-23T18:46:08.346 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_password", 2024-04-23T18:46:08.346 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.346 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.347 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.347 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.347 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.347 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.347 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.347 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Password to authenticate with", 2024-04-23T18:46:08.347 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.347 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.347 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.347 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.347 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_port": { 2024-04-23T18:46:08.347 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_port", 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "465", 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP port", 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.348 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_sender": { 2024-04-23T18:46:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_sender", 2024-04-23T18:46:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP envelope sender", 2024-04-23T18:46:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.349 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.350 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.350 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.350 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_ssl": { 2024-04-23T18:46:08.350 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_ssl", 2024-04-23T18:46:08.350 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.350 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.350 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.350 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.350 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.350 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.350 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.351 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T18:46:08.351 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.351 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.351 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.351 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.351 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_user": { 2024-04-23T18:46:08.351 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_user", 2024-04-23T18:46:08.351 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.351 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.351 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.351 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.351 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.352 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.352 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.352 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "User to authenticate as", 2024-04-23T18:46:08.352 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.352 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.352 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.352 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.352 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.352 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.352 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.352 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "balancer", 2024-04-23T18:46:08.352 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "active": { 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "active", 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T18:46:08.353 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi132.stdout: "begin_time": { 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "begin_time", 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0000", 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.354 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: "begin_weekday": { 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "begin_weekday", 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0", 2024-04-23T18:46:08.355 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "6", 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_max_iterations": { 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_max_iterations", 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "25", 2024-04-23T18:46:08.356 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "250", 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_metrics": { 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_metrics", 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.357 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T18:46:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T18:46:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T18:46:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_step": { 2024-04-23T18:46:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_step", 2024-04-23T18:46:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0.5", 2024-04-23T18:46:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0.001", 2024-04-23T18:46:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "0.999", 2024-04-23T18:46:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "aggressiveness of optimization", 2024-04-23T18:46:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T18:46:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: "end_time": { 2024-04-23T18:46:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "end_time", 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "2359", 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.360 INFO:teuthology.orchestra.run.smithi132.stdout: "end_weekday": { 2024-04-23T18:46:08.361 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "end_weekday", 2024-04-23T18:46:08.361 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:08.361 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.361 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.361 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:08.361 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0", 2024-04-23T18:46:08.361 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "6", 2024-04-23T18:46:08.361 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.361 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T18:46:08.361 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:46:08.361 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.362 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.362 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.362 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.362 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.362 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.362 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.362 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.362 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.362 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.362 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.362 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.362 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.363 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.364 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.364 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.364 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.364 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.364 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.364 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.364 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.364 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.364 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.364 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.364 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.364 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.365 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.365 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.365 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.365 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.365 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.365 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.365 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.365 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.365 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.365 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.365 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.365 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.366 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi132.stdout: "min_score": { 2024-04-23T18:46:08.367 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "min_score", 2024-04-23T18:46:08.368 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.368 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.368 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.368 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:08.368 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.368 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.368 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.368 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T18:46:08.368 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.368 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.368 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.368 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "mode": { 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mode", 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "upmap", 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "crush-compat", 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "none", 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "read", 2024-04-23T18:46:08.369 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap", 2024-04-23T18:46:08.370 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap-read" 2024-04-23T18:46:08.370 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.370 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Balancer mode", 2024-04-23T18:46:08.370 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.370 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.370 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.370 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.370 INFO:teuthology.orchestra.run.smithi132.stdout: "pool_ids": { 2024-04-23T18:46:08.370 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pool_ids", 2024-04-23T18:46:08.370 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.370 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.370 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.371 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.371 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.371 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.371 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.371 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T18:46:08.371 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.371 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.371 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.371 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.371 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:08.371 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:08.371 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.372 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.372 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.372 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:08.372 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.372 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.372 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.372 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T18:46:08.372 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.372 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.372 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.372 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.372 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap_max_deviation": { 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "upmap_max_deviation", 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.373 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.374 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap_max_optimizations": { 2024-04-23T18:46:08.374 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "upmap_max_optimizations", 2024-04-23T18:46:08.374 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:08.374 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.374 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.374 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.374 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.374 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.374 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.374 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T18:46:08.374 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.375 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.375 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.375 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.375 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.375 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.375 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.375 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cephadm", 2024-04-23T18:46:08.375 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.375 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.375 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.375 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_down_multiplier": { 2024-04-23T18:46:08.375 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_down_multiplier", 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3.0", 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.376 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_refresh_rate": { 2024-04-23T18:46:08.377 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_refresh_rate", 2024-04-23T18:46:08.377 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.377 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.377 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.377 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "20", 2024-04-23T18:46:08.377 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.377 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.377 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.377 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T18:46:08.377 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.377 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.377 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.378 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.378 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_starting_port": { 2024-04-23T18:46:08.378 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_starting_port", 2024-04-23T18:46:08.378 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.378 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.378 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.378 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "4721", 2024-04-23T18:46:08.378 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.378 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.378 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.378 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T18:46:08.379 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.379 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.379 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.379 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.379 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_ptrace": { 2024-04-23T18:46:08.379 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_ptrace", 2024-04-23T18:46:08.379 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.379 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.379 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.379 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.379 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.379 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "autotune_interval": { 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "autotune_interval", 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.380 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.381 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.381 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T18:46:08.381 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.381 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.381 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.381 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.381 INFO:teuthology.orchestra.run.smithi132.stdout: "autotune_memory_target_ratio": { 2024-04-23T18:46:08.381 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T18:46:08.381 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.381 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.381 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.381 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0.7", 2024-04-23T18:46:08.382 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.382 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.382 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.382 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T18:46:08.382 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.382 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.382 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.382 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.382 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm_log_destination": { 2024-04-23T18:46:08.382 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cephadm_log_destination", 2024-04-23T18:46:08.382 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: "file", 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: "file,syslog", 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: "syslog" 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.383 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: "cgroups_split": { 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cgroups_split", 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T18:46:08.384 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: "config_checks_enabled": { 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_checks_enabled", 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.385 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T18:46:08.386 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.386 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.386 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.386 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.386 INFO:teuthology.orchestra.run.smithi132.stdout: "config_dashboard": { 2024-04-23T18:46:08.386 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_dashboard", 2024-04-23T18:46:08.386 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.386 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.386 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.386 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.386 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.386 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.387 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.387 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T18:46:08.387 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.387 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.387 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.387 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.387 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_alertmanager": { 2024-04-23T18:46:08.387 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_alertmanager", 2024-04-23T18:46:08.387 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.387 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.387 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.387 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T18:46:08.388 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.388 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.388 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.388 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:08.388 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.388 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.388 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.388 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.388 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_base": { 2024-04-23T18:46:08.388 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_base", 2024-04-23T18:46:08.388 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.388 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Container image name, without the tag", 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_elasticsearch": { 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_elasticsearch", 2024-04-23T18:46:08.389 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.390 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.390 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.390 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T18:46:08.390 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.390 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.390 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.390 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "elasticsearch container image", 2024-04-23T18:46:08.390 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.390 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.390 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.390 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_grafana": { 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_grafana", 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.391 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.392 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.392 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_haproxy": { 2024-04-23T18:46:08.392 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_haproxy", 2024-04-23T18:46:08.392 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.392 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.392 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.392 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T18:46:08.392 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.392 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.392 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.392 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "HAproxy container image", 2024-04-23T18:46:08.392 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.393 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.393 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.393 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.393 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_agent": { 2024-04-23T18:46:08.393 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_agent", 2024-04-23T18:46:08.393 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.393 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.393 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.393 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T18:46:08.393 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.393 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.393 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.394 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger agent container image", 2024-04-23T18:46:08.394 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.394 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.394 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.394 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.394 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_collector": { 2024-04-23T18:46:08.394 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_collector", 2024-04-23T18:46:08.394 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.394 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.394 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.394 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T18:46:08.394 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.395 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.395 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.395 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger collector container image", 2024-04-23T18:46:08.395 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.395 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.395 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.395 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.395 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_query": { 2024-04-23T18:46:08.395 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_query", 2024-04-23T18:46:08.395 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.395 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.395 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.396 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T18:46:08.396 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.396 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.396 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.396 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger query container image", 2024-04-23T18:46:08.396 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.396 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.396 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.396 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.396 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_keepalived": { 2024-04-23T18:46:08.396 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_keepalived", 2024-04-23T18:46:08.396 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.397 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.397 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.397 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T18:46:08.397 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.397 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.397 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.397 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Keepalived container image", 2024-04-23T18:46:08.397 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.397 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.397 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.397 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.397 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_loki": { 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_loki", 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Loki container image", 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.398 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.399 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_node_exporter": { 2024-04-23T18:46:08.399 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_node_exporter", 2024-04-23T18:46:08.399 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.399 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.399 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.399 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T18:46:08.399 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.399 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.399 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.399 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:08.399 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.400 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.400 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.400 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.400 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_nvmeof": { 2024-04-23T18:46:08.400 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_nvmeof", 2024-04-23T18:46:08.400 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.400 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.400 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.400 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T18:46:08.400 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.400 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.400 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.401 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Nvme-of container image", 2024-04-23T18:46:08.401 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.401 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.401 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.401 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.401 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_prometheus": { 2024-04-23T18:46:08.401 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_prometheus", 2024-04-23T18:46:08.401 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.401 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.401 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.401 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T18:46:08.401 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.402 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.402 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.402 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:08.402 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.402 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.402 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.402 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.402 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_promtail": { 2024-04-23T18:46:08.402 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_promtail", 2024-04-23T18:46:08.402 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.402 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Promtail container image", 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_samba": { 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_samba", 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.403 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.404 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.404 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T18:46:08.404 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.404 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.404 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.404 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Samba/SMB container image", 2024-04-23T18:46:08.404 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.404 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.404 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.404 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.405 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_snmp_gateway": { 2024-04-23T18:46:08.405 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_snmp_gateway", 2024-04-23T18:46:08.405 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.405 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.405 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.405 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T18:46:08.405 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.405 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.405 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.405 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SNMP Gateway container image", 2024-04-23T18:46:08.405 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.405 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.406 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.406 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.406 INFO:teuthology.orchestra.run.smithi132.stdout: "container_init": { 2024-04-23T18:46:08.406 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_init", 2024-04-23T18:46:08.406 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.406 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.406 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.406 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.406 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.406 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.406 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.406 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: "daemon_cache_timeout": { 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "daemon_cache_timeout", 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.407 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: "default_cephadm_command_timeout": { 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "900", 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.408 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.409 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.409 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T18:46:08.409 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.409 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.409 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.409 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.409 INFO:teuthology.orchestra.run.smithi132.stdout: "default_registry": { 2024-04-23T18:46:08.409 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "default_registry", 2024-04-23T18:46:08.409 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.409 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.409 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.409 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io", 2024-04-23T18:46:08.410 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.410 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.410 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.410 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T18:46:08.410 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.410 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.410 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.410 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.410 INFO:teuthology.orchestra.run.smithi132.stdout: "device_cache_timeout": { 2024-04-23T18:46:08.410 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_cache_timeout", 2024-04-23T18:46:08.410 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.410 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.411 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.411 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1800", 2024-04-23T18:46:08.411 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.411 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.411 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.411 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache device inventory", 2024-04-23T18:46:08.411 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.411 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.411 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.411 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.411 INFO:teuthology.orchestra.run.smithi132.stdout: "device_enhanced_scan": { 2024-04-23T18:46:08.411 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_enhanced_scan", 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.412 INFO:teuthology.orchestra.run.smithi132.stdout: "facts_cache_timeout": { 2024-04-23T18:46:08.413 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "facts_cache_timeout", 2024-04-23T18:46:08.413 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.413 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.413 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.413 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:08.413 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.413 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.413 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.413 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache host facts data", 2024-04-23T18:46:08.413 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.413 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.414 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.414 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.414 INFO:teuthology.orchestra.run.smithi132.stdout: "grafana_dashboards_path": { 2024-04-23T18:46:08.414 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "grafana_dashboards_path", 2024-04-23T18:46:08.414 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.414 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.414 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.414 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T18:46:08.414 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.414 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.414 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.415 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T18:46:08.415 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.415 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.415 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.415 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.415 INFO:teuthology.orchestra.run.smithi132.stdout: "host_check_interval": { 2024-04-23T18:46:08.415 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "host_check_interval", 2024-04-23T18:46:08.415 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.415 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.415 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.415 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:08.415 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to perform a host check", 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: "hw_monitoring": { 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "hw_monitoring", 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.416 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.417 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.417 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.417 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.417 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T18:46:08.417 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.417 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.417 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.417 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.417 INFO:teuthology.orchestra.run.smithi132.stdout: "inventory_list_all": { 2024-04-23T18:46:08.417 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "inventory_list_all", 2024-04-23T18:46:08.417 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.417 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.418 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.418 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.418 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.418 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.418 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.418 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T18:46:08.418 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.418 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.418 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.418 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.419 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.419 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.419 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.419 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.419 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.419 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.419 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.419 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.419 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.419 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.419 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.419 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.420 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.420 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.420 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.420 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.420 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.420 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.420 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.420 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.420 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.420 INFO:teuthology.orchestra.run.smithi132.stdout: "log_refresh_metadata": { 2024-04-23T18:46:08.420 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_refresh_metadata", 2024-04-23T18:46:08.420 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.421 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.421 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.421 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.421 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.421 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.421 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.421 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T18:46:08.421 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.421 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.421 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.421 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.421 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.422 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.422 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.422 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.422 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.422 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.422 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.422 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.422 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.422 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T18:46:08.422 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.422 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.423 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.424 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.425 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.425 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.425 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.425 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.425 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.425 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.425 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.425 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.425 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.425 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.425 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.426 INFO:teuthology.orchestra.run.smithi132.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T18:46:08.426 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T18:46:08.426 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.426 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.426 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.426 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.426 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.426 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.426 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.426 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T18:46:08.426 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.426 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.427 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.427 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.427 INFO:teuthology.orchestra.run.smithi132.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T18:46:08.427 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T18:46:08.427 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.427 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.427 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.427 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "*", 2024-04-23T18:46:08.427 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.427 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.427 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.427 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: "max_count_per_host": { 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_count_per_host", 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.428 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.429 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "max number of daemons per service per host", 2024-04-23T18:46:08.429 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.429 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.429 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.429 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.429 INFO:teuthology.orchestra.run.smithi132.stdout: "max_osd_draining_count": { 2024-04-23T18:46:08.429 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_osd_draining_count", 2024-04-23T18:46:08.429 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.429 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.429 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.429 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi132.stdout: "migration_current": { 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "migration_current", 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.430 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.431 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.431 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.431 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.431 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.431 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.431 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "internal - do not modify", 2024-04-23T18:46:08.431 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.431 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.431 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.431 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.431 INFO:teuthology.orchestra.run.smithi132.stdout: "mode": { 2024-04-23T18:46:08.432 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mode", 2024-04-23T18:46:08.432 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.432 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.432 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.432 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "root", 2024-04-23T18:46:08.432 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.432 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.432 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.432 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm-package", 2024-04-23T18:46:08.432 INFO:teuthology.orchestra.run.smithi132.stdout: "root" 2024-04-23T18:46:08.432 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.432 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T18:46:08.433 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.433 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.433 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.433 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.433 INFO:teuthology.orchestra.run.smithi132.stdout: "oob_default_addr": { 2024-04-23T18:46:08.433 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "oob_default_addr", 2024-04-23T18:46:08.433 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.433 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.433 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.433 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "169.254.1.1", 2024-04-23T18:46:08.433 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.434 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.434 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.434 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T18:46:08.434 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.434 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.434 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.434 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.434 INFO:teuthology.orchestra.run.smithi132.stdout: "prometheus_alerts_path": { 2024-04-23T18:46:08.434 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prometheus_alerts_path", 2024-04-23T18:46:08.434 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.434 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.435 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.435 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T18:46:08.435 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.435 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.435 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.435 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T18:46:08.435 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.435 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.435 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.435 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.435 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_insecure": { 2024-04-23T18:46:08.435 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_insecure", 2024-04-23T18:46:08.436 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.436 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.436 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.436 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.436 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.436 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.436 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.436 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T18:46:08.436 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.436 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.436 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.436 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_password": { 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_password", 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.437 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.438 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.438 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_url": { 2024-04-23T18:46:08.438 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_url", 2024-04-23T18:46:08.438 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.438 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.438 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.438 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.438 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.438 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.438 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.438 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T18:46:08.438 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.439 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.439 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.439 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.439 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_username": { 2024-04-23T18:46:08.439 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_username", 2024-04-23T18:46:08.439 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.439 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.439 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.439 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.439 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.439 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.439 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: "secure_monitoring_stack": { 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "secure_monitoring_stack", 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.440 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi132.stdout: "service_discovery_port": { 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "service_discovery_port", 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.441 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8765", 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "cephadm service discovery port", 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: "ssh_config_file": { 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssh_config_file", 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.442 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.443 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.443 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.443 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.443 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.443 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T18:46:08.443 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.443 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.443 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.443 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.443 INFO:teuthology.orchestra.run.smithi132.stdout: "use_agent": { 2024-04-23T18:46:08.443 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "use_agent", 2024-04-23T18:46:08.443 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: "use_repo_digest": { 2024-04-23T18:46:08.444 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "use_repo_digest", 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_failed_host_check": { 2024-04-23T18:46:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_failed_host_check", 2024-04-23T18:46:08.446 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.446 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.446 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.446 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.446 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.446 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.446 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.446 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T18:46:08.446 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.446 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.446 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.446 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.447 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_stray_daemons": { 2024-04-23T18:46:08.447 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_stray_daemons", 2024-04-23T18:46:08.447 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.447 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.447 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.447 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.447 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.447 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.447 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.447 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T18:46:08.447 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.447 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.448 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.448 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.448 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_stray_hosts": { 2024-04-23T18:46:08.448 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_stray_hosts", 2024-04-23T18:46:08.448 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.448 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.448 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.448 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.448 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.448 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.448 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.448 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T18:46:08.449 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.449 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.449 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.449 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.449 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.449 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.449 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.449 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crash", 2024-04-23T18:46:08.449 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.449 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.449 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.449 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.450 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.450 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.450 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.450 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.450 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.450 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.450 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.450 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.450 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.450 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.450 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.451 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.452 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.453 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.453 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.453 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.453 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.453 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.453 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.453 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.453 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.453 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.453 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.453 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.454 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.454 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.454 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.454 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.454 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.454 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.454 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.454 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.454 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.454 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.454 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: "retain_interval": { 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "retain_interval", 2024-04-23T18:46:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.456 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.456 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.456 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "31536000", 2024-04-23T18:46:08.456 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.456 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.456 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.456 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T18:46:08.456 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.456 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.456 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.456 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.457 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_recent_interval": { 2024-04-23T18:46:08.457 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_recent_interval", 2024-04-23T18:46:08.457 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.457 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.457 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.457 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1209600", 2024-04-23T18:46:08.457 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.457 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.457 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.457 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T18:46:08.457 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.457 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.458 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.458 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.458 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.458 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.458 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.458 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "dashboard", 2024-04-23T18:46:08.458 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.458 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.458 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.458 INFO:teuthology.orchestra.run.smithi132.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T18:46:08.458 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T18:46:08.459 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.459 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.459 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.459 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.459 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.459 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.459 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.459 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.459 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.459 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.459 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.460 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.461 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.462 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.462 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.462 INFO:teuthology.orchestra.run.smithi132.stdout: "AUDIT_API_ENABLED": { 2024-04-23T18:46:08.462 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T18:46:08.462 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.462 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.462 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.462 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.462 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.462 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.463 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.464 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.465 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T18:46:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T18:46:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.466 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.467 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T18:46:08.468 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T18:46:08.469 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T18:46:08.470 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.470 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.470 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.470 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.470 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.470 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.470 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.470 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.470 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.470 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.470 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.470 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.471 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T18:46:08.471 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T18:46:08.471 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.471 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.471 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.471 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.471 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.471 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.471 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.471 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.471 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.471 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.473 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.474 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.474 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.474 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.474 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T18:46:08.474 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T18:46:08.474 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.474 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.474 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.474 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-04-23T18:46:08.474 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.474 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.474 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.475 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_URL": { 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_URL", 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.476 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.477 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.477 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.477 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.477 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.477 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.477 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.477 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T18:46:08.477 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T18:46:08.477 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.477 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.477 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.477 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-04-23T18:46:08.478 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.478 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.478 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.478 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.478 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.478 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.478 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.478 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.478 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T18:46:08.478 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T18:46:08.478 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T18:46:08.479 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.480 INFO:teuthology.orchestra.run.smithi132.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T18:46:08.481 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T18:46:08.481 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.481 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.481 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.481 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.481 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.481 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.481 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.481 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.481 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.481 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.481 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.482 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.482 INFO:teuthology.orchestra.run.smithi132.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T18:46:08.482 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T18:46:08.482 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.482 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.482 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.482 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.482 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.482 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.482 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.482 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.482 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "[]", 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.483 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.484 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.484 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.484 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.484 INFO:teuthology.orchestra.run.smithi132.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T18:46:08.484 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T18:46:08.484 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.484 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.484 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.484 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "{}", 2024-04-23T18:46:08.484 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.484 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.484 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.485 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.486 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.487 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.489 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.489 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.489 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.489 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.489 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.489 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.489 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T18:46:08.489 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T18:46:08.489 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.490 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.490 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.490 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.490 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.490 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.490 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.490 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.490 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.490 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.490 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.490 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.490 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.492 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T18:46:08.492 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T18:46:08.492 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.492 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.492 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.492 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.492 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.492 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.492 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.492 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.492 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.493 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.493 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.493 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.493 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T18:46:08.493 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T18:46:08.493 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.493 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.493 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.493 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.493 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.493 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.493 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T18:46:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T18:46:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.496 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.496 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.496 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.496 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.496 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.496 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.496 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.496 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.496 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.496 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.496 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T18:46:08.497 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T18:46:08.497 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.497 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.497 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.497 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.497 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.497 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.497 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.497 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.497 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.497 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.497 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.498 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.498 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T18:46:08.498 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T18:46:08.498 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.498 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.498 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.498 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T18:46:08.498 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.498 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.498 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.498 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.498 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.499 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.499 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.499 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.499 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T18:46:08.499 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T18:46:08.499 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.499 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.499 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.499 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.499 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.499 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.499 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8", 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.500 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.501 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.501 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.501 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.501 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.501 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.501 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.501 INFO:teuthology.orchestra.run.smithi132.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T18:46:08.501 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T18:46:08.501 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.501 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.501 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "45", 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T18:46:08.502 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.503 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.503 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.503 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.503 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.503 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.503 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.503 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.503 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.503 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.503 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.503 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.503 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.504 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.505 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T18:46:08.505 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T18:46:08.505 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.505 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.505 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.505 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.505 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.505 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.505 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.505 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.505 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.505 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.506 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.506 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.506 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T18:46:08.506 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T18:46:08.506 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.506 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.506 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.506 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.506 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.506 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.506 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.506 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.507 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.508 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.509 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.510 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.510 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.510 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.510 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.510 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.510 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.510 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.510 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T18:46:08.510 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T18:46:08.510 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.510 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.510 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: "cross_origin_url": { 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cross_origin_url", 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.511 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.512 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.512 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.512 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.512 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.512 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.512 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.512 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.512 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.512 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.512 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.512 INFO:teuthology.orchestra.run.smithi132.stdout: "crt_file": { 2024-04-23T18:46:08.513 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crt_file", 2024-04-23T18:46:08.513 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.513 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.513 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.513 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.513 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.513 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.513 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.513 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.513 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.513 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.513 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.514 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.514 INFO:teuthology.orchestra.run.smithi132.stdout: "debug": { 2024-04-23T18:46:08.514 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "debug", 2024-04-23T18:46:08.514 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.514 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.514 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.514 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.514 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.514 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.514 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.514 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable/disable debug options", 2024-04-23T18:46:08.515 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.515 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.515 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.515 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.515 INFO:teuthology.orchestra.run.smithi132.stdout: "jwt_token_ttl": { 2024-04-23T18:46:08.515 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "jwt_token_ttl", 2024-04-23T18:46:08.515 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.515 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.515 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.515 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "28800", 2024-04-23T18:46:08.515 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.515 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "key_file": { 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "key_file", 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.516 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.517 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.519 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.519 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.519 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.519 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.519 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.519 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.519 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.519 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.519 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.519 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.519 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.519 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.520 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.521 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.522 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.523 INFO:teuthology.orchestra.run.smithi132.stdout: "motd": { 2024-04-23T18:46:08.523 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "motd", 2024-04-23T18:46:08.523 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.523 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.523 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.523 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.523 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.523 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.523 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.523 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "The message of the day", 2024-04-23T18:46:08.523 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.523 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.524 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.524 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.524 INFO:teuthology.orchestra.run.smithi132.stdout: "redirect_resolve_ip_addr": { 2024-04-23T18:46:08.524 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T18:46:08.524 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.524 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.524 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.524 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.524 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.524 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.524 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "::", 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.525 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.526 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.526 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.526 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.526 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.526 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.526 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-04-23T18:46:08.526 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-04-23T18:46:08.526 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.526 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.526 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.526 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8080", 2024-04-23T18:46:08.526 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl": { 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl", 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.527 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl_server_port": { 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl_server_port", 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.528 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8443", 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_behaviour": { 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_behaviour", 2024-04-23T18:46:08.529 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.530 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.530 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.530 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "redirect", 2024-04-23T18:46:08.530 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.530 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.530 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.530 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.530 INFO:teuthology.orchestra.run.smithi132.stdout: "redirect" 2024-04-23T18:46:08.530 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.530 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.530 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.531 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.531 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.531 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.531 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_error_status_code": { 2024-04-23T18:46:08.531 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_error_status_code", 2024-04-23T18:46:08.531 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.531 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.531 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.531 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "500", 2024-04-23T18:46:08.531 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "400", 2024-04-23T18:46:08.531 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "599", 2024-04-23T18:46:08.531 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.532 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.532 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.532 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.532 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.532 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.532 INFO:teuthology.orchestra.run.smithi132.stdout: "url_prefix": { 2024-04-23T18:46:08.532 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "url_prefix", 2024-04-23T18:46:08.532 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.532 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.532 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.532 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.532 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.533 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.533 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.533 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.533 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.533 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.533 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.533 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.533 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.533 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.533 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.533 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "devicehealth", 2024-04-23T18:46:08.533 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.534 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.534 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.534 INFO:teuthology.orchestra.run.smithi132.stdout: "enable_monitoring": { 2024-04-23T18:46:08.534 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enable_monitoring", 2024-04-23T18:46:08.534 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.534 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.534 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.534 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.534 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.534 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.534 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.535 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "monitor device health metrics", 2024-04-23T18:46:08.535 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.535 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.535 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.535 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.535 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.535 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.535 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.535 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.535 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.535 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.535 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.536 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.537 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.538 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.539 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.539 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.539 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.539 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.539 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.539 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.539 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.539 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.539 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.539 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.539 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.539 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.540 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.540 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.540 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.540 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.540 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.540 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.540 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.540 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.540 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.540 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.540 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.540 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.541 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.541 INFO:teuthology.orchestra.run.smithi132.stdout: "mark_out_threshold": { 2024-04-23T18:46:08.541 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mark_out_threshold", 2024-04-23T18:46:08.541 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.541 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.541 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.541 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "2419200", 2024-04-23T18:46:08.541 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.541 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.541 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.541 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: "pool_name": { 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pool_name", 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "device_health_metrics", 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.542 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.543 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T18:46:08.543 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.543 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.543 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.543 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.543 INFO:teuthology.orchestra.run.smithi132.stdout: "retention_period": { 2024-04-23T18:46:08.543 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "retention_period", 2024-04-23T18:46:08.543 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.543 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.543 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.543 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15552000", 2024-04-23T18:46:08.543 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.544 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.544 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.544 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long to retain device health metrics", 2024-04-23T18:46:08.544 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.544 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.544 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.544 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.544 INFO:teuthology.orchestra.run.smithi132.stdout: "scrape_frequency": { 2024-04-23T18:46:08.544 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "scrape_frequency", 2024-04-23T18:46:08.544 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.544 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.544 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.545 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "86400", 2024-04-23T18:46:08.545 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.545 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.545 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.545 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T18:46:08.545 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.545 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.545 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.545 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.545 INFO:teuthology.orchestra.run.smithi132.stdout: "self_heal": { 2024-04-23T18:46:08.545 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "self_heal", 2024-04-23T18:46:08.545 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.546 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.547 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.548 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_threshold": { 2024-04-23T18:46:08.548 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_threshold", 2024-04-23T18:46:08.548 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.548 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.548 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.548 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "7257600", 2024-04-23T18:46:08.548 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.548 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.548 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.548 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T18:46:08.548 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.548 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.549 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.549 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.549 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.549 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.549 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.549 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "diskprediction_local", 2024-04-23T18:46:08.549 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.549 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.549 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.549 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.549 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.550 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.550 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.550 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.550 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.550 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.550 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.550 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.550 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.550 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.550 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.550 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.550 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.551 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.551 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.551 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.551 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.551 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.551 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.551 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.551 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.551 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.551 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.551 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.551 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.552 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.552 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.552 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.552 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.552 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.552 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.552 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.552 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.552 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.552 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.552 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.552 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.553 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.553 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.553 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.553 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.553 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.553 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.553 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.553 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.553 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.553 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.553 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.553 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.554 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.555 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.555 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.555 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.555 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.555 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.555 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.555 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.555 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.555 INFO:teuthology.orchestra.run.smithi132.stdout: "predict_interval": { 2024-04-23T18:46:08.555 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "predict_interval", 2024-04-23T18:46:08.555 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.555 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "86400", 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: "predictor_model": { 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "predictor_model", 2024-04-23T18:46:08.556 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.557 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.557 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.557 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "prophetstor", 2024-04-23T18:46:08.557 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.557 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.557 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.557 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.557 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.557 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.557 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.557 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.557 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:08.558 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:08.558 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.558 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.558 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.558 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:08.558 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.558 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.558 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.558 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.558 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.558 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.558 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "influx", 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": false, 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "influxdb python module not found", 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: "batch_size": { 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "batch_size", 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.559 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5000", 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: "database": { 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "database", 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.560 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "ceph", 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: "hostname": { 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "hostname", 2024-04-23T18:46:08.561 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.562 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.562 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.562 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.562 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.562 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.562 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.562 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB server hostname", 2024-04-23T18:46:08.562 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.562 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.562 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.562 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "30", 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "5", 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.563 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.564 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.564 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.564 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.564 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.564 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.564 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.564 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.564 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.564 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.564 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.564 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.564 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.565 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.565 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.565 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.565 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.565 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.565 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.565 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.565 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.565 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.565 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.565 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.565 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.566 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.567 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.567 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.567 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.567 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.567 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.567 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.567 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.567 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.567 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.567 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.567 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.567 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.568 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.568 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.568 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.568 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.568 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.568 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.568 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.568 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.568 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.568 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.568 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.568 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.569 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.569 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.569 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.569 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.569 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.569 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.569 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.569 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.569 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.569 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.569 INFO:teuthology.orchestra.run.smithi132.stdout: "password": { 2024-04-23T18:46:08.570 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "password", 2024-04-23T18:46:08.570 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.570 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.570 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.570 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.570 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.570 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.570 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.570 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "password of InfluxDB server user", 2024-04-23T18:46:08.570 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.570 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.571 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.571 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.571 INFO:teuthology.orchestra.run.smithi132.stdout: "port": { 2024-04-23T18:46:08.571 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "port", 2024-04-23T18:46:08.571 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.571 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.571 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.571 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8086", 2024-04-23T18:46:08.571 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.571 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.571 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.571 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB server port", 2024-04-23T18:46:08.572 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.572 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.572 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.572 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.572 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl": { 2024-04-23T18:46:08.572 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl", 2024-04-23T18:46:08.572 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.572 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.572 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.572 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "false", 2024-04-23T18:46:08.572 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.572 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "threads": { 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "threads", 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-04-23T18:46:08.573 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "32", 2024-04-23T18:46:08.574 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.574 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T18:46:08.574 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.574 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.574 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.574 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.574 INFO:teuthology.orchestra.run.smithi132.stdout: "username": { 2024-04-23T18:46:08.574 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "username", 2024-04-23T18:46:08.574 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.574 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.574 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.574 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.575 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.575 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.575 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.575 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "username of InfluxDB server user", 2024-04-23T18:46:08.575 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.575 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.575 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.575 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.575 INFO:teuthology.orchestra.run.smithi132.stdout: "verify_ssl": { 2024-04-23T18:46:08.575 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "verify_ssl", 2024-04-23T18:46:08.575 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.575 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.576 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.576 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "true", 2024-04-23T18:46:08.576 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.576 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.576 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.576 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:46:08.576 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.576 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.576 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.576 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.576 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.576 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.577 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.577 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "insights", 2024-04-23T18:46:08.577 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.577 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.577 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.577 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.577 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.577 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.577 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.577 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.577 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.577 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.578 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.579 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.579 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.579 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.579 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.579 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.579 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.579 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.579 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.579 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.579 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.579 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.579 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.580 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.580 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.580 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.580 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.580 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.580 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.580 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.580 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.580 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.580 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.580 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.581 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.582 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.582 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.582 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.582 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.582 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.582 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.582 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.582 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.582 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.582 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.582 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.582 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.583 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.583 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.583 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.583 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.583 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.583 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "iostat", 2024-04-23T18:46:08.583 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.583 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.583 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.583 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.583 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.584 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.585 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.586 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.586 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.586 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.586 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.586 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.586 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.586 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.586 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.586 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.586 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.586 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.586 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.587 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.588 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.588 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.588 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.588 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.588 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.588 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.588 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.588 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.588 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.588 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.588 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.589 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.589 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.589 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.589 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.589 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.589 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.589 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.589 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.589 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "k8sevents", 2024-04-23T18:46:08.589 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.589 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.590 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.590 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_event_retention_days": { 2024-04-23T18:46:08.590 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ceph_event_retention_days", 2024-04-23T18:46:08.590 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.590 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.590 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.590 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "7", 2024-04-23T18:46:08.590 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.590 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.590 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.590 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T18:46:08.590 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "config_check_secs": { 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_check_secs", 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "10", 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T18:46:08.591 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.592 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.592 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.592 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.592 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.592 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.592 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.592 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.592 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.592 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.592 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.592 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.592 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.593 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.593 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.593 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.593 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.593 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.593 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.593 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.593 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.593 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.593 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.593 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.593 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.594 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.595 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.595 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.595 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.595 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.595 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.595 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.595 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.595 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.595 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.595 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.595 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.596 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.596 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.596 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.596 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.596 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.596 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.596 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.596 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.596 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.596 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.596 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.596 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.597 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.597 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.597 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.597 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.597 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.597 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.597 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.597 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.597 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.597 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.597 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.597 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.598 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.598 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.598 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.598 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.598 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "localpool", 2024-04-23T18:46:08.598 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.598 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.598 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.598 INFO:teuthology.orchestra.run.smithi132.stdout: "failure_domain": { 2024-04-23T18:46:08.598 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "failure_domain", 2024-04-23T18:46:08.598 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.598 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "host", 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "failure domain for any created local pool", 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.599 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.600 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.600 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.600 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.600 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.600 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.600 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.600 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.600 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.600 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.600 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.600 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.600 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.601 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.601 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.601 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.601 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.601 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.601 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.601 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.601 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.601 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.602 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.602 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.602 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.602 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.602 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.602 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.602 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.602 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.602 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.602 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.602 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.602 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.603 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.603 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.603 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.603 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.603 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.603 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.603 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.603 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.603 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.603 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.603 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.603 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.604 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.604 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.604 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.604 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.604 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.604 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.604 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.604 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.604 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.604 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.604 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.604 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.605 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.605 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.605 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.605 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.605 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.605 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.605 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.605 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.605 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.605 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.605 INFO:teuthology.orchestra.run.smithi132.stdout: "min_size": { 2024-04-23T18:46:08.605 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "min_size", 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default min_size for any created local pool", 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.606 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.607 INFO:teuthology.orchestra.run.smithi132.stdout: "num_rep": { 2024-04-23T18:46:08.607 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "num_rep", 2024-04-23T18:46:08.607 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.607 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.607 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.607 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3", 2024-04-23T18:46:08.607 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.607 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.607 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.607 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default replica count for any created local pool", 2024-04-23T18:46:08.607 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.607 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_num": { 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pg_num", 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "128", 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T18:46:08.608 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: "prefix": { 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prefix", 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.609 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "name prefix for any created local pool", 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: "subtree": { 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "subtree", 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "rack", 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.610 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.611 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T18:46:08.611 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T18:46:08.611 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.611 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.611 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.611 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.611 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.611 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.611 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mds_autoscaler", 2024-04-23T18:46:08.611 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.611 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.611 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.612 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.613 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.614 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.615 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.615 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.615 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.615 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.615 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.615 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.615 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.615 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.615 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.615 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.615 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.615 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.616 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mirroring", 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.617 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.618 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.618 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.618 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.618 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.618 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.618 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.618 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.618 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.618 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.618 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.618 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.618 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.619 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.619 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.619 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.619 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.619 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.619 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.619 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.619 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.619 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.619 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.619 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.620 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.620 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.620 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.620 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.620 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.620 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.620 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.620 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.620 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.620 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.620 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.621 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.621 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.621 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.621 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.621 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.621 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.621 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.621 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.621 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.621 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.621 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.621 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.622 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.622 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.622 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.622 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.622 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.622 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.622 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.622 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.622 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.622 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.622 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.622 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.623 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "nfs", 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.624 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.625 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.625 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.625 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.625 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.625 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.625 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.625 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.625 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.625 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.625 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.625 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.625 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.626 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.626 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.626 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.626 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.626 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.626 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.626 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.626 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.626 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.626 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.627 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.628 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.629 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "orchestrator", 2024-04-23T18:46:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.630 INFO:teuthology.orchestra.run.smithi132.stdout: "fail_fs": { 2024-04-23T18:46:08.631 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "fail_fs", 2024-04-23T18:46:08.631 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.631 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.631 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.631 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.631 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.631 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.631 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.631 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T18:46:08.631 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.631 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.631 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.632 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.632 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.632 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.632 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.632 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.632 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.632 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.632 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.632 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.632 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.632 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.632 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.633 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.633 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.633 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.633 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.633 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.633 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.633 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.633 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.633 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.633 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.633 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.633 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.634 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.634 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.634 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.634 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.634 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.634 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.634 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.634 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.634 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.634 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.634 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.634 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.635 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.636 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.637 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.637 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.637 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.637 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.637 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.637 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.637 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.637 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.637 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator": { 2024-04-23T18:46:08.637 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "orchestrator", 2024-04-23T18:46:08.637 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.637 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm", 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: "rook", 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: "test_orchestrator" 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Orchestrator backend", 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.638 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.639 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.639 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.639 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.639 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.639 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "osd_perf_query", 2024-04-23T18:46:08.639 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.639 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.639 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.639 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.639 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.639 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.640 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.640 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.640 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.640 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.640 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.640 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.640 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.640 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.640 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.640 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.640 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.640 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.641 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.641 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.641 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.641 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.641 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.641 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.641 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.641 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.641 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.641 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.641 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.641 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.642 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.642 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.642 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.642 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.642 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.642 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.642 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.642 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.642 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.642 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.642 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.643 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.643 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.643 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.643 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.643 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.643 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.643 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.643 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.643 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.643 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.643 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.643 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.644 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.644 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.644 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.644 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.644 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.644 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.644 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.644 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.644 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.644 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.644 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.644 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "osd_support", 2024-04-23T18:46:08.645 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.646 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.647 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.649 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.649 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.649 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.649 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.649 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.649 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.649 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.649 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.649 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.649 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.649 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.649 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.650 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.650 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.650 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.650 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.650 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.650 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.650 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.650 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.650 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.650 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.650 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.650 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.651 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.651 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.651 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.651 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.651 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.651 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.651 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.651 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.651 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.651 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.651 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pg_autoscaler", 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.652 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.653 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.653 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.653 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.653 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.653 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.653 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.653 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.653 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.653 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.653 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.653 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.653 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.654 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.655 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.655 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.655 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.655 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.655 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.655 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.655 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.655 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.655 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.655 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.655 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.655 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.656 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:08.657 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:08.658 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.658 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.658 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.658 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:08.658 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.658 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.658 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.658 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.658 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.658 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.658 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.658 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.659 INFO:teuthology.orchestra.run.smithi132.stdout: "threshold": { 2024-04-23T18:46:08.659 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "threshold", 2024-04-23T18:46:08.659 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.659 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.659 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.659 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3.0", 2024-04-23T18:46:08.659 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1.0", 2024-04-23T18:46:08.659 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.659 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.659 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "scaling threshold", 2024-04-23T18:46:08.659 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T18:46:08.659 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "progress", 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_pg_recovery_event": { 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_pg_recovery_event", 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.660 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.661 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.661 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.661 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.661 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.661 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.661 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T18:46:08.661 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.661 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.661 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.661 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.661 INFO:teuthology.orchestra.run.smithi132.stdout: "enabled": { 2024-04-23T18:46:08.661 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enabled", 2024-04-23T18:46:08.662 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.662 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.662 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.662 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.662 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.662 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.662 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.662 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.662 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.662 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.662 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.662 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.663 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.663 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.663 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.663 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.663 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.663 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.663 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.663 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.663 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.663 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.663 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.663 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.665 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.666 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.666 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.666 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.666 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.666 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.666 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.666 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.666 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.666 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.666 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.666 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.667 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.667 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.667 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.667 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.667 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.667 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.667 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.667 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.667 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.667 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.667 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.667 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.668 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.668 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.668 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.668 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.668 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.668 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.668 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.668 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.668 INFO:teuthology.orchestra.run.smithi132.stdout: "max_completed_events": { 2024-04-23T18:46:08.668 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_completed_events", 2024-04-23T18:46:08.668 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "50", 2024-04-23T18:46:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "number of past completed events to remember", 2024-04-23T18:46:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-04-23T18:46:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long the module is going to sleep", 2024-04-23T18:46:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prometheus", 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "cache": { 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cache", 2024-04-23T18:46:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "exclude_perf_counters": { 2024-04-23T18:46:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "exclude_perf_counters", 2024-04-23T18:46:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T18:46:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T18:46:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_stats_pools": { 2024-04-23T18:46:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_stats_pools", 2024-04-23T18:46:08.680 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.680 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.680 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.680 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.680 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.680 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.680 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.680 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.680 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.680 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.680 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.680 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "300", 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.681 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.682 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.682 INFO:teuthology.orchestra.run.smithi132.stdout: "scrape_interval": { 2024-04-23T18:46:08.682 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "scrape_interval", 2024-04-23T18:46:08.682 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.682 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.682 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.682 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15.0", 2024-04-23T18:46:08.682 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.682 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.682 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.682 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.682 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.683 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.683 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.683 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.683 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-04-23T18:46:08.683 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-04-23T18:46:08.683 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.683 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.683 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.683 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "::", 2024-04-23T18:46:08.683 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.683 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "9283", 2024-04-23T18:46:08.684 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: "stale_cache_strategy": { 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "stale_cache_strategy", 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.685 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "log", 2024-04-23T18:46:08.686 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.686 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.686 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.686 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.686 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.686 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.686 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.686 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.686 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_behaviour": { 2024-04-23T18:46:08.686 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_behaviour", 2024-04-23T18:46:08.686 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.686 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.687 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.687 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "default", 2024-04-23T18:46:08.687 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.687 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.687 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.687 INFO:teuthology.orchestra.run.smithi132.stdout: "default", 2024-04-23T18:46:08.687 INFO:teuthology.orchestra.run.smithi132.stdout: "error" 2024-04-23T18:46:08.687 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.687 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.687 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.687 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.687 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_error_status_code": { 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_error_status_code", 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "500", 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "400", 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "599", 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.688 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_support", 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.689 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.690 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.690 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.690 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.690 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.690 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.690 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.690 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.690 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.690 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.690 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.690 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.690 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.691 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.691 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.691 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.691 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.691 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.691 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.691 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.691 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.691 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.693 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.693 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.693 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.693 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.693 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.693 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.693 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.693 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.693 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.694 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.694 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.694 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.694 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.694 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.694 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.694 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.694 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.694 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.694 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.694 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.695 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.695 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.695 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.695 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.695 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.695 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.695 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.695 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.695 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.695 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.695 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.695 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.696 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.696 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.696 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.696 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.696 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.696 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.696 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.696 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.696 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.696 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.696 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.697 INFO:teuthology.orchestra.run.smithi132.stdout: "max_concurrent_snap_create": { 2024-04-23T18:46:08.697 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_concurrent_snap_create", 2024-04-23T18:46:08.697 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.697 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.697 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.697 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.697 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.697 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.697 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.697 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.697 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.697 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.698 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.698 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.698 INFO:teuthology.orchestra.run.smithi132.stdout: "mirror_snapshot_schedule": { 2024-04-23T18:46:08.698 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T18:46:08.698 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.698 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.698 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.698 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.698 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.698 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.698 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: "trash_purge_schedule": { 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "trash_purge_schedule", 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.699 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.700 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.700 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.700 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.700 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.700 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.700 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.700 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.700 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.700 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.700 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "restful", 2024-04-23T18:46:08.700 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: "enable_auth": { 2024-04-23T18:46:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enable_auth", 2024-04-23T18:46:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.701 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.702 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.702 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.702 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.702 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.702 INFO:teuthology.orchestra.run.smithi132.stdout: "key_file": { 2024-04-23T18:46:08.702 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "key_file", 2024-04-23T18:46:08.702 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.702 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.702 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.702 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.702 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.702 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.703 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.704 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.704 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.704 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.704 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.704 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.704 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.704 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.704 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.704 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.704 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.704 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.704 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.705 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.706 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.706 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.706 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.706 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.706 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.706 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.706 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.706 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.706 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.706 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.706 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.706 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.707 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.708 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.708 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.708 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.708 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.708 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.708 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.708 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.708 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.708 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.708 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.708 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.708 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.709 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.710 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.710 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-04-23T18:46:08.710 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-04-23T18:46:08.710 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.710 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.710 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.710 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.710 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.710 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.710 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.710 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.710 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rgw", 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.711 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.712 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.712 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.712 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.712 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.712 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.712 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.712 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.712 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.712 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.712 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.712 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.712 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.713 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.714 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.714 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.714 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.714 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.714 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.714 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.714 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.714 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.714 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.714 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.714 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.714 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.715 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.716 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rook", 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.717 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.718 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.719 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.719 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.719 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.719 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.719 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.719 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.719 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.719 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.719 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.719 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.719 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.719 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.720 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.721 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.722 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.722 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.722 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.722 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.722 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.722 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.722 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.722 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.722 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.722 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.722 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.722 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "storage_class": { 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "storage_class", 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "local", 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T18:46:08.723 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "selftest", 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.724 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.725 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.725 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.725 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.725 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.725 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.725 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.725 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.725 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.725 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.725 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.725 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.726 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.727 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.728 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.729 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.730 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.730 INFO:teuthology.orchestra.run.smithi132.stdout: "roption1": { 2024-04-23T18:46:08.730 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "roption1", 2024-04-23T18:46:08.730 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.730 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.730 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.730 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.730 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.730 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.730 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.730 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.730 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "roption2": { 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "roption2", 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "xyz", 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.731 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption1": { 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption1", 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.732 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption2": { 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption2", 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.733 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.734 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.734 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.734 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.734 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.734 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.734 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption3": { 2024-04-23T18:46:08.734 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption3", 2024-04-23T18:46:08.734 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.734 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.734 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.734 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption4": { 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption4", 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.735 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption5": { 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption5", 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.736 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption6": { 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption6", 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.737 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.738 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.738 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.738 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.738 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.738 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.738 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.738 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.738 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.738 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.738 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption7": { 2024-04-23T18:46:08.738 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption7", 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "42", 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: "testkey": { 2024-04-23T18:46:08.740 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testkey", 2024-04-23T18:46:08.740 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.740 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.740 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.740 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.740 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.740 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.740 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.740 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.740 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.740 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.740 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: "testlkey": { 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testlkey", 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.741 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.742 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.742 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.742 INFO:teuthology.orchestra.run.smithi132.stdout: "testnewline": { 2024-04-23T18:46:08.742 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testnewline", 2024-04-23T18:46:08.742 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.742 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.742 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.742 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.742 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.742 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.742 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.743 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.743 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.743 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.743 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.743 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.743 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.743 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.743 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.743 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snap_schedule", 2024-04-23T18:46:08.743 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.743 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.743 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_m_granularity": { 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_m_granularity", 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.744 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.745 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.745 INFO:teuthology.orchestra.run.smithi132.stdout: "dump_on_update": { 2024-04-23T18:46:08.745 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "dump_on_update", 2024-04-23T18:46:08.745 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.745 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.745 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.745 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.745 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.745 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.745 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.745 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "dump database to debug log on update", 2024-04-23T18:46:08.745 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.746 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.746 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.746 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.746 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.746 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.746 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.746 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.746 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.746 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.746 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.746 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.746 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.747 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.748 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.748 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.748 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.748 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.748 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.748 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.748 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.748 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.748 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.748 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.748 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.748 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.749 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.750 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.750 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.750 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.750 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.750 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.750 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.750 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.750 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.750 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.750 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.750 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.750 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.751 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.751 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.751 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.751 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.751 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.751 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.751 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.751 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.751 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.751 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.751 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.751 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "stats", 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.752 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.753 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.754 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.754 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.754 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.754 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.754 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.754 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.754 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.754 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.754 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.754 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.754 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.754 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.755 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.755 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.755 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.755 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.755 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.755 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.755 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.755 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.755 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.755 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.755 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.755 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.756 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.757 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.758 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.758 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.758 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.758 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "status", 2024-04-23T18:46:08.758 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.758 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.758 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.758 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.758 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.758 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.758 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.758 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.759 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.759 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.759 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.759 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.759 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.759 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.759 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.759 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.759 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.759 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.759 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.759 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.760 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.760 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.760 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.760 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.760 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.760 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.760 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.760 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.760 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.760 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.760 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.760 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.761 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.761 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.761 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.761 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.761 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.761 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.761 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.761 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.761 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.761 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.761 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.761 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.762 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.763 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.763 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.763 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.763 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.763 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.763 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.763 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.763 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.763 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.763 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.763 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.763 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "telegraf", 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: "address": { 2024-04-23T18:46:08.764 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "address", 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.765 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:08.766 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:08.766 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.766 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.766 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.766 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15", 2024-04-23T18:46:08.766 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.766 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.766 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.766 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.766 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.766 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.767 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.768 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.769 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.769 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.769 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.769 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.769 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.769 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.769 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.769 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.769 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.769 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.769 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.769 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.770 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.770 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.770 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.770 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.770 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.770 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.770 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.770 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.770 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.770 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.770 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.771 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.771 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.771 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.771 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.771 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.771 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.771 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.771 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.771 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.771 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.771 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.772 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.772 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.772 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.772 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.772 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.772 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.772 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.772 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.772 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.772 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.772 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "telemetry", 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_basic": { 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_basic", 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.773 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_crash": { 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_crash", 2024-04-23T18:46:08.774 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.775 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.775 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.775 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.775 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.775 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.775 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.775 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T18:46:08.775 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.775 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.775 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.775 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.776 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_device": { 2024-04-23T18:46:08.776 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_device", 2024-04-23T18:46:08.776 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.776 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.776 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.776 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.776 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.776 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.776 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.776 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T18:46:08.776 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.776 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.777 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.777 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.777 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_ident": { 2024-04-23T18:46:08.777 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_ident", 2024-04-23T18:46:08.777 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.777 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.777 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.777 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.777 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.777 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.777 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.777 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T18:46:08.778 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.778 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.778 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.778 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.778 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_perf": { 2024-04-23T18:46:08.778 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_perf", 2024-04-23T18:46:08.778 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.778 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.778 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.778 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.778 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.779 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.779 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.779 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T18:46:08.779 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.779 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.779 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.779 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.779 INFO:teuthology.orchestra.run.smithi132.stdout: "contact": { 2024-04-23T18:46:08.779 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "contact", 2024-04-23T18:46:08.779 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.779 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.780 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.780 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.780 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.780 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.780 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.780 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.780 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.780 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.780 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.780 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.780 INFO:teuthology.orchestra.run.smithi132.stdout: "description": { 2024-04-23T18:46:08.780 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "description", 2024-04-23T18:46:08.781 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.781 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.781 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.781 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.781 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.781 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.781 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.781 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.781 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.781 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.781 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.781 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi132.stdout: "device_url": { 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_url", 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.782 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.783 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.783 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.783 INFO:teuthology.orchestra.run.smithi132.stdout: "enabled": { 2024-04-23T18:46:08.783 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enabled", 2024-04-23T18:46:08.783 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.783 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.783 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.783 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.783 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.783 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.783 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.783 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "24", 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "8", 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.784 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.785 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.785 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.785 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.785 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.785 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.785 INFO:teuthology.orchestra.run.smithi132.stdout: "last_opt_revision": { 2024-04-23T18:46:08.785 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "last_opt_revision", 2024-04-23T18:46:08.785 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.785 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.785 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.785 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1", 2024-04-23T18:46:08.785 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: "leaderboard": { 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "leaderboard", 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.786 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.787 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.787 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.787 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.787 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.787 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.787 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.787 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.787 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.787 INFO:teuthology.orchestra.run.smithi132.stdout: "leaderboard_description": { 2024-04-23T18:46:08.787 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "leaderboard_description", 2024-04-23T18:46:08.787 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.787 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.788 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.788 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.788 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.788 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.788 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.788 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.788 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.788 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.788 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.788 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.788 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.788 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.790 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.790 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.790 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.790 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.790 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.790 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.790 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.790 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.790 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.790 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.790 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.791 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.792 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.792 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.792 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.792 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.792 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.792 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.792 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.792 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.792 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.792 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.792 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.792 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.793 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.793 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.793 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.793 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.793 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.793 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.793 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.793 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.793 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.793 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.793 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.793 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.794 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.794 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.794 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.794 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.794 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.794 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.794 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.794 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.794 INFO:teuthology.orchestra.run.smithi132.stdout: "organization": { 2024-04-23T18:46:08.794 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "organization", 2024-04-23T18:46:08.794 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: "proxy": { 2024-04-23T18:46:08.795 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "proxy", 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.796 INFO:teuthology.orchestra.run.smithi132.stdout: "url": { 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "url", 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.797 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.798 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.798 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.798 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.798 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "test_orchestrator", 2024-04-23T18:46:08.798 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.798 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.798 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.798 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.798 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.798 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.798 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.798 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.799 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.799 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.799 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.799 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.799 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.799 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.799 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.799 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.799 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.799 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.799 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.799 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.800 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.801 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.801 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.801 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.801 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.801 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.801 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.801 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.801 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.801 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.801 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.801 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.801 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.802 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.802 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.802 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.802 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.802 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.802 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.802 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.802 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.802 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.802 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.802 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.803 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "volumes", 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.804 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.805 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.806 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.806 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.806 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.806 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.806 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.806 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.806 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.806 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.806 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.806 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.806 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.806 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.807 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.807 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.807 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.807 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.807 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.807 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.807 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.807 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.807 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.807 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.807 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.807 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.808 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.809 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.809 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.809 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.809 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.809 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.809 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.809 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.809 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.809 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.809 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.809 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.809 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.810 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.810 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.810 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.810 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.810 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.810 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.810 INFO:teuthology.orchestra.run.smithi132.stdout: "max_concurrent_clones": { 2024-04-23T18:46:08.810 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_concurrent_clones", 2024-04-23T18:46:08.810 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.810 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.810 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "4", 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "periodic_async_work": { 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "periodic_async_work", 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.811 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.812 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.812 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.812 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.812 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.812 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Periodically check for async work", 2024-04-23T18:46:08.812 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.812 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.812 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.812 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.812 INFO:teuthology.orchestra.run.smithi132.stdout: "snapshot_clone_delay": { 2024-04-23T18:46:08.812 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snapshot_clone_delay", 2024-04-23T18:46:08.812 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: "snapshot_clone_no_wait": { 2024-04-23T18:46:08.813 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T18:46:08.814 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.814 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.814 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.814 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.814 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.814 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.814 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.814 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T18:46:08.814 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.814 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.814 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.814 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix", 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: "discovery_interval": { 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "discovery_interval", 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.815 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "100", 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: "identifier": { 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "identifier", 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.816 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.817 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.817 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.817 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.817 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.817 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.817 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.817 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.817 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.817 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.817 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:08.817 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:08.818 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.818 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.818 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.818 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:08.818 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.818 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.818 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.818 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.818 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.818 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.818 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.819 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.820 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.821 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.821 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.821 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.821 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.821 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.821 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.821 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.821 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.821 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.821 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.821 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.821 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.822 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.822 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.822 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.822 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.822 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.822 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.822 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.822 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.822 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.822 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.822 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.822 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.823 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_host": { 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_host", 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.824 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.825 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.825 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.825 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.825 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.825 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.825 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.825 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.825 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.825 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.825 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_port": { 2024-04-23T18:46:08.825 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_port", 2024-04-23T18:46:08.825 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.826 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.826 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.826 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10051", 2024-04-23T18:46:08.826 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.826 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.826 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.826 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.826 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.826 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.826 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.826 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.826 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_sender": { 2024-04-23T18:46:08.827 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_sender", 2024-04-23T18:46:08.827 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.827 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.827 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.827 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T18:46:08.827 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.827 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.827 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.827 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.827 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.827 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.827 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: "modules": [ 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm", 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: "dashboard", 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: "iostat", 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: "nfs", 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: "restful", 2024-04-23T18:46:08.828 INFO:teuthology.orchestra.run.smithi132.stdout: "snap_schedule" 2024-04-23T18:46:08.829 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.829 INFO:teuthology.orchestra.run.smithi132.stdout: "available_modules": [ 2024-04-23T18:46:08.829 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.829 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "alerts", 2024-04-23T18:46:08.829 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.829 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.829 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.829 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:08.829 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:08.829 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.829 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.829 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.830 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:08.830 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.830 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.830 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.830 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T18:46:08.830 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.830 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.830 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.830 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.830 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.830 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.830 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.831 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.831 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.831 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.831 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.831 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.831 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.831 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.831 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.831 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.831 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.831 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.831 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.832 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.832 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.832 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.832 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.832 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.832 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.832 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.832 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.832 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.832 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.832 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.832 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.833 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.834 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.834 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.834 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.834 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.834 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.834 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.834 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.834 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.834 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.834 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.834 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.834 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.835 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.835 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.835 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.835 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.835 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.835 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.835 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.835 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.835 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.835 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.835 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.835 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_destination": { 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_destination", 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.836 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.837 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.837 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.837 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Email address to send alerts to", 2024-04-23T18:46:08.837 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.837 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.837 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.837 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.837 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_from_name": { 2024-04-23T18:46:08.837 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_from_name", 2024-04-23T18:46:08.837 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.837 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.837 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.838 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "Ceph", 2024-04-23T18:46:08.838 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.838 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.838 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.838 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Email From: name", 2024-04-23T18:46:08.838 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.838 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.838 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.838 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.838 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_host": { 2024-04-23T18:46:08.838 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_host", 2024-04-23T18:46:08.838 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP server", 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_password": { 2024-04-23T18:46:08.839 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_password", 2024-04-23T18:46:08.840 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.840 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.840 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.840 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.840 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.840 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.840 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.840 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Password to authenticate with", 2024-04-23T18:46:08.840 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.840 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.840 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.841 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.841 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_port": { 2024-04-23T18:46:08.841 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_port", 2024-04-23T18:46:08.841 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.841 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.841 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.841 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "465", 2024-04-23T18:46:08.841 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.841 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.841 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.841 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP port", 2024-04-23T18:46:08.841 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_sender": { 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_sender", 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.842 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.843 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP envelope sender", 2024-04-23T18:46:08.843 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.843 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.843 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.843 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.843 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_ssl": { 2024-04-23T18:46:08.843 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_ssl", 2024-04-23T18:46:08.843 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.843 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.843 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.843 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.843 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.844 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.844 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.844 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T18:46:08.844 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.844 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.844 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.844 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.844 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_user": { 2024-04-23T18:46:08.844 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_user", 2024-04-23T18:46:08.844 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.844 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.844 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "User to authenticate as", 2024-04-23T18:46:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.846 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "balancer", 2024-04-23T18:46:08.846 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.846 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.846 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.846 INFO:teuthology.orchestra.run.smithi132.stdout: "active": { 2024-04-23T18:46:08.846 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "active", 2024-04-23T18:46:08.846 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.846 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.846 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.846 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.846 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.847 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.847 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.847 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T18:46:08.847 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.847 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.847 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.847 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.847 INFO:teuthology.orchestra.run.smithi132.stdout: "begin_time": { 2024-04-23T18:46:08.847 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "begin_time", 2024-04-23T18:46:08.847 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.848 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.848 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.848 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0000", 2024-04-23T18:46:08.848 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.848 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.848 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.848 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T18:46:08.848 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:46:08.848 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.848 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: "begin_weekday": { 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "begin_weekday", 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0", 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "6", 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:46:08.849 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_max_iterations": { 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_max_iterations", 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "25", 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "250", 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.850 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T18:46:08.851 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.851 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.851 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.851 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.851 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_metrics": { 2024-04-23T18:46:08.851 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_metrics", 2024-04-23T18:46:08.851 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.851 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.851 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.851 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T18:46:08.851 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.852 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.852 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.852 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T18:46:08.852 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T18:46:08.852 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.852 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.852 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.852 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_step": { 2024-04-23T18:46:08.852 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_step", 2024-04-23T18:46:08.852 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.852 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.852 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0.5", 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0.001", 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "0.999", 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "aggressiveness of optimization", 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: "end_time": { 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "end_time", 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.853 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.854 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.854 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "2359", 2024-04-23T18:46:08.854 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.854 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.854 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.854 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T18:46:08.854 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:46:08.854 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.854 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.854 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.854 INFO:teuthology.orchestra.run.smithi132.stdout: "end_weekday": { 2024-04-23T18:46:08.854 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "end_weekday", 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0", 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "6", 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.855 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.856 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.857 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.858 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.859 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.859 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.859 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.859 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.859 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.859 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.859 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.859 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.859 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.859 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.859 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.859 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.860 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: "min_score": { 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "min_score", 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:08.861 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.862 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.862 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.862 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T18:46:08.862 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.862 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.862 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.862 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.862 INFO:teuthology.orchestra.run.smithi132.stdout: "mode": { 2024-04-23T18:46:08.862 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mode", 2024-04-23T18:46:08.862 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.862 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.862 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "upmap", 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: "crush-compat", 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: "none", 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: "read", 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap", 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap-read" 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Balancer mode", 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.863 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: "pool_ids": { 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pool_ids", 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T18:46:08.864 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.865 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap_max_deviation": { 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "upmap_max_deviation", 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-04-23T18:46:08.866 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.867 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.867 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T18:46:08.867 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T18:46:08.867 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.867 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.867 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.867 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap_max_optimizations": { 2024-04-23T18:46:08.867 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "upmap_max_optimizations", 2024-04-23T18:46:08.867 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:08.867 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.867 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.867 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cephadm", 2024-04-23T18:46:08.868 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_down_multiplier": { 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_down_multiplier", 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3.0", 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T18:46:08.869 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.870 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.870 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.870 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.870 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_refresh_rate": { 2024-04-23T18:46:08.870 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_refresh_rate", 2024-04-23T18:46:08.870 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.870 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.870 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.870 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "20", 2024-04-23T18:46:08.870 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.870 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.870 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.871 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T18:46:08.871 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.871 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.871 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.871 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.871 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_starting_port": { 2024-04-23T18:46:08.871 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_starting_port", 2024-04-23T18:46:08.871 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.871 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.871 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.871 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "4721", 2024-04-23T18:46:08.871 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_ptrace": { 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_ptrace", 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.872 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.873 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.873 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.873 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.873 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T18:46:08.873 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T18:46:08.873 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.873 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.873 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.873 INFO:teuthology.orchestra.run.smithi132.stdout: "autotune_interval": { 2024-04-23T18:46:08.873 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "autotune_interval", 2024-04-23T18:46:08.873 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.873 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.874 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.874 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:08.874 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.874 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.874 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.874 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T18:46:08.874 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.874 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.874 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.874 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.874 INFO:teuthology.orchestra.run.smithi132.stdout: "autotune_memory_target_ratio": { 2024-04-23T18:46:08.874 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T18:46:08.875 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:08.875 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.875 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.875 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0.7", 2024-04-23T18:46:08.875 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.875 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.875 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.875 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T18:46:08.875 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.875 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.875 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.875 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm_log_destination": { 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cephadm_log_destination", 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: "file", 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: "file,syslog", 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: "syslog" 2024-04-23T18:46:08.876 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: "cgroups_split": { 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cgroups_split", 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.877 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.878 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.878 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T18:46:08.878 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.878 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.878 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.878 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.878 INFO:teuthology.orchestra.run.smithi132.stdout: "config_checks_enabled": { 2024-04-23T18:46:08.878 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_checks_enabled", 2024-04-23T18:46:08.878 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.878 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.878 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.879 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.879 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.879 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.879 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.879 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T18:46:08.879 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.879 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.879 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.879 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.879 INFO:teuthology.orchestra.run.smithi132.stdout: "config_dashboard": { 2024-04-23T18:46:08.879 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_dashboard", 2024-04-23T18:46:08.879 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.880 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.880 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.880 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.880 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.880 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.880 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.880 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T18:46:08.880 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.880 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.880 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.880 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.880 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_alertmanager": { 2024-04-23T18:46:08.881 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_alertmanager", 2024-04-23T18:46:08.881 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.881 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.881 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.881 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T18:46:08.881 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.881 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.881 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.881 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:08.881 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.881 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.882 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.882 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.882 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_base": { 2024-04-23T18:46:08.882 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_base", 2024-04-23T18:46:08.882 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.882 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.882 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.882 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T18:46:08.882 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.882 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.882 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.882 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Container image name, without the tag", 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_elasticsearch": { 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_elasticsearch", 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.883 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.884 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "elasticsearch container image", 2024-04-23T18:46:08.884 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.884 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.884 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.884 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.884 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_grafana": { 2024-04-23T18:46:08.884 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_grafana", 2024-04-23T18:46:08.884 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.884 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.884 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.884 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T18:46:08.884 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.885 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.885 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.885 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:08.885 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.885 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.885 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.885 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.885 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_haproxy": { 2024-04-23T18:46:08.885 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_haproxy", 2024-04-23T18:46:08.885 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.885 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.885 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T18:46:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "HAproxy container image", 2024-04-23T18:46:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_agent": { 2024-04-23T18:46:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_agent", 2024-04-23T18:46:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger agent container image", 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_collector": { 2024-04-23T18:46:08.887 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_collector", 2024-04-23T18:46:08.888 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.888 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.888 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.888 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T18:46:08.888 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.888 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.888 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.888 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger collector container image", 2024-04-23T18:46:08.888 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.888 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.888 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.888 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.889 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_query": { 2024-04-23T18:46:08.889 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_query", 2024-04-23T18:46:08.889 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.889 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.889 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.889 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T18:46:08.889 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.889 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.889 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.889 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger query container image", 2024-04-23T18:46:08.889 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.889 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.890 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.890 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.890 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_keepalived": { 2024-04-23T18:46:08.890 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_keepalived", 2024-04-23T18:46:08.890 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.890 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.890 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.890 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T18:46:08.890 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.890 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.890 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.891 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Keepalived container image", 2024-04-23T18:46:08.891 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.891 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.891 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.891 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.891 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_loki": { 2024-04-23T18:46:08.891 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_loki", 2024-04-23T18:46:08.891 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.891 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.891 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.891 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T18:46:08.891 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.892 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.892 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.892 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Loki container image", 2024-04-23T18:46:08.892 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.892 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.892 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.892 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.892 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_node_exporter": { 2024-04-23T18:46:08.892 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_node_exporter", 2024-04-23T18:46:08.892 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.892 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.893 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.893 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T18:46:08.893 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.893 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.893 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.893 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:08.893 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.893 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.893 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.893 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.893 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_nvmeof": { 2024-04-23T18:46:08.893 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_nvmeof", 2024-04-23T18:46:08.894 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.894 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.894 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.894 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T18:46:08.894 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.894 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.894 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.894 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Nvme-of container image", 2024-04-23T18:46:08.894 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.894 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.894 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.895 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.895 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_prometheus": { 2024-04-23T18:46:08.895 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_prometheus", 2024-04-23T18:46:08.895 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.895 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.895 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.895 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T18:46:08.895 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.895 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.895 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.895 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-04-23T18:46:08.895 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.896 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.896 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.896 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.896 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_promtail": { 2024-04-23T18:46:08.896 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_promtail", 2024-04-23T18:46:08.896 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.896 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.896 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.896 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T18:46:08.896 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.896 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.896 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.897 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Promtail container image", 2024-04-23T18:46:08.897 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.897 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.897 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.897 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.897 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_samba": { 2024-04-23T18:46:08.897 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_samba", 2024-04-23T18:46:08.897 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.897 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.897 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.897 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T18:46:08.897 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.898 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.898 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.898 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Samba/SMB container image", 2024-04-23T18:46:08.898 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.898 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.898 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.898 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.898 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_snmp_gateway": { 2024-04-23T18:46:08.898 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_snmp_gateway", 2024-04-23T18:46:08.898 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.898 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.898 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.901 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T18:46:08.901 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.901 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.901 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.901 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SNMP Gateway container image", 2024-04-23T18:46:08.901 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.901 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.902 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.902 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.902 INFO:teuthology.orchestra.run.smithi132.stdout: "container_init": { 2024-04-23T18:46:08.902 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_init", 2024-04-23T18:46:08.902 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.902 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.902 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.902 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.902 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.902 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.902 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.902 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T18:46:08.903 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.903 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.903 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.903 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.903 INFO:teuthology.orchestra.run.smithi132.stdout: "daemon_cache_timeout": { 2024-04-23T18:46:08.903 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "daemon_cache_timeout", 2024-04-23T18:46:08.903 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.903 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.903 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.903 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:08.903 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.903 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: "default_cephadm_command_timeout": { 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "900", 2024-04-23T18:46:08.904 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.905 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.905 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.905 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T18:46:08.905 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.905 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.905 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.905 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.905 INFO:teuthology.orchestra.run.smithi132.stdout: "default_registry": { 2024-04-23T18:46:08.905 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "default_registry", 2024-04-23T18:46:08.905 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.905 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.905 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.906 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io", 2024-04-23T18:46:08.906 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.906 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.906 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.906 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T18:46:08.906 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.906 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.906 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.906 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.906 INFO:teuthology.orchestra.run.smithi132.stdout: "device_cache_timeout": { 2024-04-23T18:46:08.906 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_cache_timeout", 2024-04-23T18:46:08.906 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.907 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.907 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.907 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1800", 2024-04-23T18:46:08.907 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.907 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.907 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.907 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache device inventory", 2024-04-23T18:46:08.907 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.907 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.907 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.907 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.907 INFO:teuthology.orchestra.run.smithi132.stdout: "device_enhanced_scan": { 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_enhanced_scan", 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.908 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "facts_cache_timeout": { 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "facts_cache_timeout", 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache host facts data", 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.909 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.910 INFO:teuthology.orchestra.run.smithi132.stdout: "grafana_dashboards_path": { 2024-04-23T18:46:08.910 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "grafana_dashboards_path", 2024-04-23T18:46:08.910 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.910 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.910 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.910 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T18:46:08.910 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.910 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.910 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.910 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T18:46:08.910 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.911 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.911 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.911 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.911 INFO:teuthology.orchestra.run.smithi132.stdout: "host_check_interval": { 2024-04-23T18:46:08.911 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "host_check_interval", 2024-04-23T18:46:08.911 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.911 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.911 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.911 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:08.911 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.911 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.911 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to perform a host check", 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "hw_monitoring": { 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "hw_monitoring", 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.912 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.913 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T18:46:08.913 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.913 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.913 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.913 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.913 INFO:teuthology.orchestra.run.smithi132.stdout: "inventory_list_all": { 2024-04-23T18:46:08.913 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "inventory_list_all", 2024-04-23T18:46:08.913 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.913 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.913 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.913 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.914 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.915 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.915 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.915 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.915 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.915 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.915 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.915 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.915 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.915 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.915 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.915 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.916 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.916 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.916 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.916 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.916 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.916 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.916 INFO:teuthology.orchestra.run.smithi132.stdout: "log_refresh_metadata": { 2024-04-23T18:46:08.916 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_refresh_metadata", 2024-04-23T18:46:08.916 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.916 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.916 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.916 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.917 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.917 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.917 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.917 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T18:46:08.917 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.917 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.917 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.917 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.917 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.917 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.917 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.918 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.918 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.918 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.918 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.918 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.918 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.918 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T18:46:08.918 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.918 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.918 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.918 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.918 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.919 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.919 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.919 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.919 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.919 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.919 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.919 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.919 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.919 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.919 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.919 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.919 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.920 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.921 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.921 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.921 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.921 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.921 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.921 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.921 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.921 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.921 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.921 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.921 INFO:teuthology.orchestra.run.smithi132.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T18:46:08.921 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T18:46:08.922 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.922 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.922 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.922 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.922 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.922 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.922 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.922 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T18:46:08.922 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.922 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.922 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.922 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.923 INFO:teuthology.orchestra.run.smithi132.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T18:46:08.923 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T18:46:08.923 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.923 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.923 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.923 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "*", 2024-04-23T18:46:08.923 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.923 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.923 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.923 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T18:46:08.923 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.923 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.924 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.924 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.924 INFO:teuthology.orchestra.run.smithi132.stdout: "max_count_per_host": { 2024-04-23T18:46:08.924 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_count_per_host", 2024-04-23T18:46:08.924 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.924 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.924 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.924 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.924 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.924 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.924 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.924 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "max number of daemons per service per host", 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: "max_osd_draining_count": { 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_osd_draining_count", 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.925 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.926 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T18:46:08.926 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.926 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.926 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.926 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.926 INFO:teuthology.orchestra.run.smithi132.stdout: "migration_current": { 2024-04-23T18:46:08.926 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "migration_current", 2024-04-23T18:46:08.926 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "internal - do not modify", 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: "mode": { 2024-04-23T18:46:08.927 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mode", 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "root", 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm-package", 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: "root" 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.928 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.929 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.929 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.929 INFO:teuthology.orchestra.run.smithi132.stdout: "oob_default_addr": { 2024-04-23T18:46:08.929 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "oob_default_addr", 2024-04-23T18:46:08.929 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.929 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.929 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.929 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "169.254.1.1", 2024-04-23T18:46:08.929 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.929 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.929 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.929 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T18:46:08.930 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.930 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.930 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.930 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.930 INFO:teuthology.orchestra.run.smithi132.stdout: "prometheus_alerts_path": { 2024-04-23T18:46:08.930 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prometheus_alerts_path", 2024-04-23T18:46:08.930 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.930 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.930 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.930 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T18:46:08.930 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.930 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.931 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.931 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T18:46:08.931 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.931 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.931 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.931 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.931 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_insecure": { 2024-04-23T18:46:08.931 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_insecure", 2024-04-23T18:46:08.931 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.931 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.931 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_password": { 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_password", 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.932 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.933 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.933 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.933 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.933 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.933 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.933 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T18:46:08.933 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.933 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.933 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.933 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.933 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_url": { 2024-04-23T18:46:08.934 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_url", 2024-04-23T18:46:08.934 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.934 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.934 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.934 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.934 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.934 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.934 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.934 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T18:46:08.934 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.934 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_username": { 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_username", 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T18:46:08.935 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.936 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.936 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.936 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.936 INFO:teuthology.orchestra.run.smithi132.stdout: "secure_monitoring_stack": { 2024-04-23T18:46:08.936 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "secure_monitoring_stack", 2024-04-23T18:46:08.936 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.936 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.936 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.936 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.936 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.936 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.936 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: "service_discovery_port": { 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "service_discovery_port", 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8765", 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.937 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.938 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.938 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "cephadm service discovery port", 2024-04-23T18:46:08.938 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.938 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.938 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.938 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.938 INFO:teuthology.orchestra.run.smithi132.stdout: "ssh_config_file": { 2024-04-23T18:46:08.938 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssh_config_file", 2024-04-23T18:46:08.938 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.938 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.938 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.938 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.939 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.939 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.939 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.939 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T18:46:08.939 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.939 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.939 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.939 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.939 INFO:teuthology.orchestra.run.smithi132.stdout: "use_agent": { 2024-04-23T18:46:08.939 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "use_agent", 2024-04-23T18:46:08.939 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: "use_repo_digest": { 2024-04-23T18:46:08.940 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "use_repo_digest", 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.941 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_failed_host_check": { 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_failed_host_check", 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.942 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_stray_daemons": { 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_stray_daemons", 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.943 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_stray_hosts": { 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_stray_hosts", 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crash", 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.945 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.946 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.946 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.946 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.946 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.946 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.946 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.946 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.946 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.946 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.946 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.946 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.946 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.947 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.948 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:08.949 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:08.950 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.950 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.950 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.950 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.950 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.950 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.950 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.950 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.950 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.950 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.950 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: "retain_interval": { 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "retain_interval", 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "31536000", 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.951 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_recent_interval": { 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_recent_interval", 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1209600", 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T18:46:08.952 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.953 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.953 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.953 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.953 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:08.953 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.953 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:08.953 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "dashboard", 2024-04-23T18:46:08.953 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:08.953 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:08.953 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:08.953 INFO:teuthology.orchestra.run.smithi132.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T18:46:08.953 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T18:46:08.954 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.954 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.954 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.954 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.954 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.954 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.954 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.954 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.954 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.954 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.954 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.956 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.956 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.956 INFO:teuthology.orchestra.run.smithi132.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T18:46:08.956 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T18:46:08.956 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.956 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.956 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.956 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.956 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.956 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.956 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: "AUDIT_API_ENABLED": { 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.957 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.958 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.958 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.958 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.958 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.958 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.958 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.958 INFO:teuthology.orchestra.run.smithi132.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T18:46:08.958 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T18:46:08.958 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.958 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.958 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.958 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.959 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.959 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.959 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.959 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.959 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.959 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.959 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.959 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.959 INFO:teuthology.orchestra.run.smithi132.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T18:46:08.959 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T18:46:08.959 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.959 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.960 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.960 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.960 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.960 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.960 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.960 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.960 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.960 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.960 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.960 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.960 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T18:46:08.960 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T18:46:08.961 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.961 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.961 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.961 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.961 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.961 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.961 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.961 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.961 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.961 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.961 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.961 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T18:46:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T18:46:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.963 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.964 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.964 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.964 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.964 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T18:46:08.964 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T18:46:08.964 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.964 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.964 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.964 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.964 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.964 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.965 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.965 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.965 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.965 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.965 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.965 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.965 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T18:46:08.965 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T18:46:08.965 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.965 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.965 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.965 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.966 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.966 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.966 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.966 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.966 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.966 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.966 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.966 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.966 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T18:46:08.966 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T18:46:08.966 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.966 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T18:46:08.967 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.968 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.968 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.968 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.968 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.968 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.968 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.968 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.968 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.968 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.968 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.968 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.968 INFO:teuthology.orchestra.run.smithi132.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T18:46:08.969 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T18:46:08.969 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.969 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.969 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.969 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.969 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.969 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.969 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.969 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.969 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.969 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.969 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.970 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.970 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T18:46:08.970 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T18:46:08.970 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.970 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.970 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.970 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-04-23T18:46:08.970 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.970 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.970 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.970 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.971 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.972 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.972 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.972 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.972 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.972 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.972 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_URL": { 2024-04-23T18:46:08.972 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_URL", 2024-04-23T18:46:08.972 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.972 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.972 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.972 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.972 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.973 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-04-23T18:46:08.974 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.974 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.974 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.974 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.974 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.974 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.974 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.974 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.974 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T18:46:08.974 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T18:46:08.974 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.974 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T18:46:08.975 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T18:46:08.976 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.976 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.976 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.976 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.976 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.976 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.976 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.976 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.976 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.976 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.976 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.976 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.977 INFO:teuthology.orchestra.run.smithi132.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T18:46:08.977 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T18:46:08.977 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.977 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.977 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.977 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.977 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.977 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.977 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.977 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.977 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.977 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.978 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.979 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.979 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.979 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.979 INFO:teuthology.orchestra.run.smithi132.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T18:46:08.979 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T18:46:08.979 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.979 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.979 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.979 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "[]", 2024-04-23T18:46:08.979 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.979 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.979 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.980 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.980 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.980 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.980 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.980 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.980 INFO:teuthology.orchestra.run.smithi132.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T18:46:08.980 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T18:46:08.980 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.980 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.980 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.980 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "{}", 2024-04-23T18:46:08.980 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.981 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.981 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.981 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.981 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.981 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.981 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.981 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.981 INFO:teuthology.orchestra.run.smithi132.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T18:46:08.981 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T18:46:08.981 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.981 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.981 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.982 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.982 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.982 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.982 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.982 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.982 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.982 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.982 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.982 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.982 INFO:teuthology.orchestra.run.smithi132.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T18:46:08.982 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T18:46:08.983 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.983 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.983 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.983 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.983 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.983 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.983 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.983 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.983 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.983 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.983 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.984 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.985 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.985 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.985 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T18:46:08.985 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T18:46:08.985 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.985 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.985 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.985 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.985 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.985 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.985 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.986 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.987 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.987 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.987 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.987 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.987 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.987 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.987 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.987 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T18:46:08.987 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T18:46:08.987 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.987 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T18:46:08.988 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.989 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.989 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.989 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.989 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.989 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.989 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.989 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.989 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.989 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.989 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.990 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.990 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T18:46:08.990 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T18:46:08.990 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.990 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.990 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.990 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.990 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.990 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.990 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.990 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.991 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.991 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.991 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.991 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.991 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T18:46:08.991 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T18:46:08.991 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.991 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.991 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.991 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:08.991 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.991 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.992 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.992 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.992 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.992 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.992 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.992 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.992 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T18:46:08.992 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T18:46:08.992 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:08.992 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.992 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.992 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:08.993 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.993 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.993 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.993 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.993 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.993 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.993 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.993 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.993 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T18:46:08.993 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T18:46:08.993 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.993 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.994 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.994 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T18:46:08.994 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.994 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.994 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.994 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.994 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.994 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.994 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.994 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.994 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T18:46:08.994 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T18:46:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8", 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.996 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "45", 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.997 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.998 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.998 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.998 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T18:46:08.998 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T18:46:08.998 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.998 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.998 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.998 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:08.998 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.998 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.998 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:08.998 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:08.999 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.000 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.001 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.001 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.001 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.001 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.001 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.001 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.001 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T18:46:09.001 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T18:46:09.001 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.001 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.001 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.001 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T18:46:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T18:46:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.002 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T18:46:09.003 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T18:46:09.004 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T18:46:09.005 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.005 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.005 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.005 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:09.005 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.005 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.005 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.005 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.005 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.005 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.005 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.005 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.006 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T18:46:09.006 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T18:46:09.006 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.006 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.006 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.006 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-04-23T18:46:09.006 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.006 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.006 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.006 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.006 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.006 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.007 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.007 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.007 INFO:teuthology.orchestra.run.smithi132.stdout: "cross_origin_url": { 2024-04-23T18:46:09.007 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cross_origin_url", 2024-04-23T18:46:09.007 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.007 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.007 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.007 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.007 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.007 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.007 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.007 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: "crt_file": { 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crt_file", 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.008 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.009 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.009 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.009 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.009 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.009 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.009 INFO:teuthology.orchestra.run.smithi132.stdout: "debug": { 2024-04-23T18:46:09.009 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "debug", 2024-04-23T18:46:09.009 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.009 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.009 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.009 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.009 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.010 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.010 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.010 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable/disable debug options", 2024-04-23T18:46:09.010 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.010 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.010 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.010 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.010 INFO:teuthology.orchestra.run.smithi132.stdout: "jwt_token_ttl": { 2024-04-23T18:46:09.010 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "jwt_token_ttl", 2024-04-23T18:46:09.010 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.010 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.011 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.011 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "28800", 2024-04-23T18:46:09.011 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.011 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.011 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.011 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.011 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.011 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.011 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.011 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.011 INFO:teuthology.orchestra.run.smithi132.stdout: "key_file": { 2024-04-23T18:46:09.011 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "key_file", 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.012 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.013 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.013 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.013 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.013 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.013 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.013 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.013 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.013 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.013 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.013 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.013 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.014 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.014 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.014 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.014 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.014 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.014 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.014 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.014 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.014 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.014 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.014 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.014 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.015 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.016 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.016 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.016 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.016 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.016 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.016 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.016 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.016 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.016 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.016 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.016 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.016 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.017 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.017 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.017 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.017 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.017 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.017 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.017 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.017 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.017 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.017 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.017 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.017 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.018 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.018 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.018 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.018 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.018 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.018 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.018 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.018 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.018 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.018 INFO:teuthology.orchestra.run.smithi132.stdout: "motd": { 2024-04-23T18:46:09.018 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "motd", 2024-04-23T18:46:09.019 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.019 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.019 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.019 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.019 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.019 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.019 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.019 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "The message of the day", 2024-04-23T18:46:09.019 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.019 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.020 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.020 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.020 INFO:teuthology.orchestra.run.smithi132.stdout: "redirect_resolve_ip_addr": { 2024-04-23T18:46:09.020 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T18:46:09.020 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.020 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.020 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.020 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.020 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.020 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.020 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.021 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.021 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.021 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.021 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.021 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.021 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-04-23T18:46:09.021 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-04-23T18:46:09.021 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.021 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.021 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.021 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "::", 2024-04-23T18:46:09.021 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.022 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.022 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.022 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.022 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.022 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.022 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.022 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.022 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-04-23T18:46:09.022 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-04-23T18:46:09.022 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.022 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.022 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.023 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8080", 2024-04-23T18:46:09.023 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.023 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.023 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.023 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.023 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.023 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.023 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.023 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.023 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl": { 2024-04-23T18:46:09.023 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl", 2024-04-23T18:46:09.023 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl_server_port": { 2024-04-23T18:46:09.024 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl_server_port", 2024-04-23T18:46:09.025 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.025 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.025 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.025 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8443", 2024-04-23T18:46:09.025 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.025 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.025 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.025 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.025 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.025 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.025 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.025 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_behaviour": { 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_behaviour", 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "redirect", 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: "redirect" 2024-04-23T18:46:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_error_status_code": { 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_error_status_code", 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "500", 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "400", 2024-04-23T18:46:09.027 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "599", 2024-04-23T18:46:09.028 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.028 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.028 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.028 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.028 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.028 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.028 INFO:teuthology.orchestra.run.smithi132.stdout: "url_prefix": { 2024-04-23T18:46:09.028 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "url_prefix", 2024-04-23T18:46:09.028 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.028 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.028 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.028 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.029 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.029 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.029 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.029 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.029 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.029 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.029 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.029 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.029 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.029 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.029 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.029 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "devicehealth", 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "enable_monitoring": { 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enable_monitoring", 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.030 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "monitor device health metrics", 2024-04-23T18:46:09.031 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.031 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.031 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.031 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.031 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.031 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.031 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.031 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.031 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.031 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.031 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.032 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.032 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.032 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.032 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.032 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.032 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.032 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.032 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.032 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.032 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.032 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.033 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.034 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.034 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.034 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.034 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.034 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.034 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.034 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.034 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.034 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.034 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.034 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.035 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.036 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.036 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.036 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.036 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.036 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.036 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.036 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.036 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.036 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.036 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.036 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.036 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: "mark_out_threshold": { 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mark_out_threshold", 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "2419200", 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.037 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T18:46:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: "pool_name": { 2024-04-23T18:46:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pool_name", 2024-04-23T18:46:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "device_health_metrics", 2024-04-23T18:46:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: "retention_period": { 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "retention_period", 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15552000", 2024-04-23T18:46:09.039 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.040 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.040 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.040 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long to retain device health metrics", 2024-04-23T18:46:09.040 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.040 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.040 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.040 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.040 INFO:teuthology.orchestra.run.smithi132.stdout: "scrape_frequency": { 2024-04-23T18:46:09.041 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "scrape_frequency", 2024-04-23T18:46:09.041 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:09.041 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.041 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.041 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "86400", 2024-04-23T18:46:09.041 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.041 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.041 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.041 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T18:46:09.041 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.041 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.041 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.042 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.042 INFO:teuthology.orchestra.run.smithi132.stdout: "self_heal": { 2024-04-23T18:46:09.042 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "self_heal", 2024-04-23T18:46:09.042 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.042 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.042 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.042 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.042 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.042 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.042 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.042 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T18:46:09.042 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.043 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.043 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.043 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.043 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:09.043 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:09.043 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:09.043 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.043 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.043 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:09.043 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.043 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.044 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.044 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T18:46:09.044 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.044 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.044 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.044 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.044 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_threshold": { 2024-04-23T18:46:09.044 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_threshold", 2024-04-23T18:46:09.044 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:09.044 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.044 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.044 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "7257600", 2024-04-23T18:46:09.045 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.045 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.045 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.045 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T18:46:09.045 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.045 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.045 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.045 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.045 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.045 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.045 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.046 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "diskprediction_local", 2024-04-23T18:46:09.046 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.046 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.046 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.046 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.046 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.046 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.046 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.046 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.046 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.046 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.047 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.047 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.047 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.047 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.047 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.047 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.047 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.047 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.047 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.047 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.047 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.047 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.048 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.048 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.048 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.048 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.048 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.048 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.048 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.048 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.048 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.048 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.048 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.048 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.049 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.049 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.049 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.049 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.049 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.049 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.049 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.049 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.049 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.049 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.049 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.049 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.050 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.050 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.050 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.050 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.050 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.050 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.050 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.050 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.050 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.050 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.050 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.050 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.051 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.051 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.051 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.051 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.051 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.051 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.051 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.051 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.051 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.051 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.051 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.051 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.052 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.052 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.052 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.052 INFO:teuthology.orchestra.run.smithi132.stdout: "predict_interval": { 2024-04-23T18:46:09.052 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "predict_interval", 2024-04-23T18:46:09.052 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.052 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.052 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.052 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "86400", 2024-04-23T18:46:09.052 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.052 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.052 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.053 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.053 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.053 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.053 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.053 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.053 INFO:teuthology.orchestra.run.smithi132.stdout: "predictor_model": { 2024-04-23T18:46:09.053 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "predictor_model", 2024-04-23T18:46:09.053 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.053 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.053 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.053 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "prophetstor", 2024-04-23T18:46:09.053 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.054 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.054 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.054 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.054 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.054 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.054 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.054 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.054 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:09.054 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:09.054 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.054 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.054 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.055 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "influx", 2024-04-23T18:46:09.056 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": false, 2024-04-23T18:46:09.056 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "influxdb python module not found", 2024-04-23T18:46:09.056 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.056 INFO:teuthology.orchestra.run.smithi132.stdout: "batch_size": { 2024-04-23T18:46:09.056 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "batch_size", 2024-04-23T18:46:09.056 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.056 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.056 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.056 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5000", 2024-04-23T18:46:09.056 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.056 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.056 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: "database": { 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "database", 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "ceph", 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.057 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.058 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.058 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T18:46:09.058 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.058 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.058 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.058 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.058 INFO:teuthology.orchestra.run.smithi132.stdout: "hostname": { 2024-04-23T18:46:09.058 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "hostname", 2024-04-23T18:46:09.058 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.058 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.058 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.058 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB server hostname", 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.059 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "30", 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "5", 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.060 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.061 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.061 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.061 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.061 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.061 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.061 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.061 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.061 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.061 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.061 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.061 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.061 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.063 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.064 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.064 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.064 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.064 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.064 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.064 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.064 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.064 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.064 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.064 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.064 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.065 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.065 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.065 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.065 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.065 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.065 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.065 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.065 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.065 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.065 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.065 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.065 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.066 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.066 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.066 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.066 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.066 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.066 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.066 INFO:teuthology.orchestra.run.smithi132.stdout: "password": { 2024-04-23T18:46:09.066 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "password", 2024-04-23T18:46:09.066 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.066 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.066 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.066 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "password of InfluxDB server user", 2024-04-23T18:46:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: "port": { 2024-04-23T18:46:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "port", 2024-04-23T18:46:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8086", 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB server port", 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl": { 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl", 2024-04-23T18:46:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "false", 2024-04-23T18:46:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:46:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: "threads": { 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "threads", 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "32", 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.070 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.071 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.071 INFO:teuthology.orchestra.run.smithi132.stdout: "username": { 2024-04-23T18:46:09.071 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "username", 2024-04-23T18:46:09.071 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.071 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.071 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.071 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.071 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.071 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.071 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.071 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "username of InfluxDB server user", 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "verify_ssl": { 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "verify_ssl", 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "true", 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.072 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:46:09.073 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.073 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.073 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.073 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.073 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.073 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.073 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.073 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "insights", 2024-04-23T18:46:09.073 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.073 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.073 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.073 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.074 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.074 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.074 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.074 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.074 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.074 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.074 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.074 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.074 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.074 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.074 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.074 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.075 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.076 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.077 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.077 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.077 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.077 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.077 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.077 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.077 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.077 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.077 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.077 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.077 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.077 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.079 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.079 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.079 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.079 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.079 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.079 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.079 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.079 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.079 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.079 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.079 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "iostat", 2024-04-23T18:46:09.079 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.080 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.081 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.081 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.081 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.081 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.081 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.081 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.081 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.081 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.081 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.081 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.081 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.082 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.083 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.083 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.083 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.083 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.083 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.083 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.083 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.083 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.083 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.083 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.083 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.084 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.085 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.086 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "k8sevents", 2024-04-23T18:46:09.086 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.086 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.086 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.086 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_event_retention_days": { 2024-04-23T18:46:09.086 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ceph_event_retention_days", 2024-04-23T18:46:09.086 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.086 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.086 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.086 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "7", 2024-04-23T18:46:09.086 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.086 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: "config_check_secs": { 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_check_secs", 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:09.087 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "10", 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.088 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.089 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.089 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.089 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.089 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.089 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.089 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.089 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.089 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.089 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.089 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.089 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.090 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.090 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.090 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.090 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.090 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.090 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.090 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.090 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.090 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.090 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.090 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.091 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.092 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.092 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.092 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.092 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.092 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.092 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.092 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.092 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.092 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.092 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.092 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.092 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.093 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "localpool", 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: "failure_domain": { 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "failure_domain", 2024-04-23T18:46:09.094 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "host", 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "failure domain for any created local pool", 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.095 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.096 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.097 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.097 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.097 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.097 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.097 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.097 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.097 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.097 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.097 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.097 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.097 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.097 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.098 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.098 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.098 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.098 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.098 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.098 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.098 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.098 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.098 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.098 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.098 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.098 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.099 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.100 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.100 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.100 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.100 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.100 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.100 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.100 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.100 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.100 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.100 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.100 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.100 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.101 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.101 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.101 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.101 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.101 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.101 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.101 INFO:teuthology.orchestra.run.smithi132.stdout: "min_size": { 2024-04-23T18:46:09.101 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "min_size", 2024-04-23T18:46:09.101 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.101 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.101 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.101 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.102 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.102 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.102 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.102 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default min_size for any created local pool", 2024-04-23T18:46:09.102 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T18:46:09.102 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.102 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.102 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.102 INFO:teuthology.orchestra.run.smithi132.stdout: "num_rep": { 2024-04-23T18:46:09.102 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "num_rep", 2024-04-23T18:46:09.102 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.102 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.103 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.103 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3", 2024-04-23T18:46:09.103 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.103 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.103 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.103 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default replica count for any created local pool", 2024-04-23T18:46:09.103 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.103 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.103 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.105 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.105 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_num": { 2024-04-23T18:46:09.105 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pg_num", 2024-04-23T18:46:09.105 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.105 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.105 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.105 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "128", 2024-04-23T18:46:09.105 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.105 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.106 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.106 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T18:46:09.106 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.106 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.106 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.106 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.106 INFO:teuthology.orchestra.run.smithi132.stdout: "prefix": { 2024-04-23T18:46:09.106 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prefix", 2024-04-23T18:46:09.106 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.106 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.106 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.106 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "name prefix for any created local pool", 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: "subtree": { 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "subtree", 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.107 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "rack", 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.108 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mds_autoscaler", 2024-04-23T18:46:09.109 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.109 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.109 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.109 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.109 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.109 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.109 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.109 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.109 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.109 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.109 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.109 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.110 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.110 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.110 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.110 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.110 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.110 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.110 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.110 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.110 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.110 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.110 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.110 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.111 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.111 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.111 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.111 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.111 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.111 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.111 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.111 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.111 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.111 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.111 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.111 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.112 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.113 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.114 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mirroring", 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.115 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.116 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.117 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.118 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.118 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.118 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.118 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.118 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.118 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.118 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.118 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.118 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.118 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.118 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.118 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.119 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.119 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.119 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.119 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.119 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.119 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.119 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.119 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.119 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.119 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.119 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.119 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.120 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "nfs", 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.121 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.122 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.123 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.124 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.125 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.126 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.126 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.126 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.126 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.126 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.126 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.126 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.126 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.126 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.126 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.126 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "orchestrator", 2024-04-23T18:46:09.126 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "fail_fs": { 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "fail_fs", 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T18:46:09.127 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.128 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.129 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.129 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.129 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.129 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.129 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.129 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.129 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.129 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.129 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.129 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.129 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.129 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.130 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.130 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.130 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.130 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.130 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.130 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.130 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.130 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.130 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.130 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.130 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.130 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.131 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.131 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.131 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.131 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.131 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.131 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.131 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.131 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.131 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.131 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.131 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.131 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.132 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.132 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.132 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.132 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.132 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.132 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.132 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.132 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.132 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.132 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.132 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.132 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator": { 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "orchestrator", 2024-04-23T18:46:09.133 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm", 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: "rook", 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: "test_orchestrator" 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Orchestrator backend", 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.134 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "osd_perf_query", 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.135 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.136 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.137 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.137 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.137 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.137 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.137 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.137 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.137 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.137 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.137 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.137 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.137 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.137 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.138 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.138 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.138 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.138 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.138 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.138 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.138 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.138 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.138 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.138 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.138 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.138 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.139 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.140 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.140 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.140 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.140 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.140 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.140 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.140 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.140 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.140 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.140 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.140 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.140 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.141 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.141 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.141 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.141 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.141 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "osd_support", 2024-04-23T18:46:09.141 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.141 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.141 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.141 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.141 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.141 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.141 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.142 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.142 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.142 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.142 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.142 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.142 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.142 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.142 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.142 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.142 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.142 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.143 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.143 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.143 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.143 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.143 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.143 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.143 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.143 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.143 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.143 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.143 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.143 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.144 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.145 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.146 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pg_autoscaler", 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.147 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.148 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.149 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.150 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.150 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.150 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.150 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.150 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.150 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.150 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.150 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.150 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.150 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.150 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.150 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.151 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.152 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.153 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.153 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.153 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:09.153 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:09.153 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:09.153 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.153 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.153 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:09.153 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.153 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.153 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.153 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: "threshold": { 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "threshold", 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3.0", 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1.0", 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.154 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "scaling threshold", 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "progress", 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.155 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.156 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_pg_recovery_event": { 2024-04-23T18:46:09.156 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_pg_recovery_event", 2024-04-23T18:46:09.156 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.156 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.156 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.156 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.156 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.156 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.156 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.156 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T18:46:09.156 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.156 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: "enabled": { 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enabled", 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.157 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.158 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.158 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.158 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.158 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.158 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.158 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.158 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.158 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.158 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.158 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.158 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.159 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.159 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.159 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.159 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.159 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.159 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.159 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.159 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.159 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.159 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.159 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.159 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.160 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.161 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.162 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.162 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.162 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.162 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.162 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.162 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.162 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.162 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.162 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.162 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.162 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "max_completed_events": { 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_completed_events", 2024-04-23T18:46:09.163 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "50", 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "number of past completed events to remember", 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-04-23T18:46:09.164 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-04-23T18:46:09.165 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:09.165 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.165 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.165 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-04-23T18:46:09.165 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.165 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.165 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.165 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long the module is going to sleep", 2024-04-23T18:46:09.165 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.165 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.165 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.165 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.166 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.166 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.166 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.166 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prometheus", 2024-04-23T18:46:09.166 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.166 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.166 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.166 INFO:teuthology.orchestra.run.smithi132.stdout: "cache": { 2024-04-23T18:46:09.166 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cache", 2024-04-23T18:46:09.166 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.166 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: "exclude_perf_counters": { 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "exclude_perf_counters", 2024-04-23T18:46:09.167 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.168 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.168 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.168 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.168 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.168 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.168 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.168 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T18:46:09.168 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T18:46:09.168 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.168 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.168 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.169 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.170 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.171 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.171 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.171 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.171 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.171 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.171 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.171 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.171 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.171 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.171 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.171 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.171 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.172 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.173 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_stats_pools": { 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_stats_pools", 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.174 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.175 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.175 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.175 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.175 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.175 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.175 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.175 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.175 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T18:46:09.175 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T18:46:09.175 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.175 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.175 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.176 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "300", 2024-04-23T18:46:09.176 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.176 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.176 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.176 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.176 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.176 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.176 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.176 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.176 INFO:teuthology.orchestra.run.smithi132.stdout: "scrape_interval": { 2024-04-23T18:46:09.176 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "scrape_interval", 2024-04-23T18:46:09.176 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15.0", 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-04-23T18:46:09.177 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-04-23T18:46:09.178 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.178 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.178 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.178 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "::", 2024-04-23T18:46:09.178 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.178 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.178 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.178 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T18:46:09.178 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.178 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.178 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.178 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "9283", 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.179 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.180 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.180 INFO:teuthology.orchestra.run.smithi132.stdout: "stale_cache_strategy": { 2024-04-23T18:46:09.180 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "stale_cache_strategy", 2024-04-23T18:46:09.180 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.180 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.180 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.180 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "log", 2024-04-23T18:46:09.180 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.180 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.180 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.180 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.180 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_behaviour": { 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_behaviour", 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "default", 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.181 INFO:teuthology.orchestra.run.smithi132.stdout: "default", 2024-04-23T18:46:09.182 INFO:teuthology.orchestra.run.smithi132.stdout: "error" 2024-04-23T18:46:09.182 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.182 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.182 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.182 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.182 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.182 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.182 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_error_status_code": { 2024-04-23T18:46:09.182 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_error_status_code", 2024-04-23T18:46:09.182 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.182 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.182 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "500", 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "400", 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "599", 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.183 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_support", 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.184 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.185 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.185 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.185 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.185 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.185 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.185 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.185 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.185 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.185 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.185 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.185 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.185 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.187 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.188 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.188 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.188 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.188 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.188 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.188 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.188 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.188 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.188 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.188 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.188 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.188 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.189 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.189 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.189 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.189 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.189 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.189 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.189 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.189 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.189 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.189 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.189 INFO:teuthology.orchestra.run.smithi132.stdout: "max_concurrent_snap_create": { 2024-04-23T18:46:09.189 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_concurrent_snap_create", 2024-04-23T18:46:09.190 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.190 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.190 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.190 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-04-23T18:46:09.190 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.190 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.190 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.190 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.190 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.190 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.190 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.191 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.191 INFO:teuthology.orchestra.run.smithi132.stdout: "mirror_snapshot_schedule": { 2024-04-23T18:46:09.191 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T18:46:09.191 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.191 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.191 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.191 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.191 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.191 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.191 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.191 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.191 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.192 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.192 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.192 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.192 INFO:teuthology.orchestra.run.smithi132.stdout: "trash_purge_schedule": { 2024-04-23T18:46:09.192 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "trash_purge_schedule", 2024-04-23T18:46:09.192 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.192 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.192 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.192 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.192 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.192 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.192 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "restful", 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.193 INFO:teuthology.orchestra.run.smithi132.stdout: "enable_auth": { 2024-04-23T18:46:09.194 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enable_auth", 2024-04-23T18:46:09.194 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.194 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.194 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.194 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.194 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.194 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.194 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.194 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.194 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.194 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.194 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: "key_file": { 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "key_file", 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.195 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.196 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.197 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.197 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.197 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.197 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.197 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.197 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.197 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.197 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.197 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.197 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.197 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.197 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.198 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.199 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.200 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.201 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.202 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.203 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.204 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rgw", 2024-04-23T18:46:09.204 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.204 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.204 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.204 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.204 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.204 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.204 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.204 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.204 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.204 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.204 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.205 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.205 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.205 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.205 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.205 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.205 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.205 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.205 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.205 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.205 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.205 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.205 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.206 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.207 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.208 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.209 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.210 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.210 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rook", 2024-04-23T18:46:09.210 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.210 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.210 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.210 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.210 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.210 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.210 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.210 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.210 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.211 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.212 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.212 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.212 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.212 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.212 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.212 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.212 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.212 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.212 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.212 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.212 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.212 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.213 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.213 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.213 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.213 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.213 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.213 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.213 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.213 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.213 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.213 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.213 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.213 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.214 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.214 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.214 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.214 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.214 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.214 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.214 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.214 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.214 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.214 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.214 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.214 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.215 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "storage_class": { 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "storage_class", 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "local", 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T18:46:09.216 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "selftest", 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.217 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.218 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.218 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.218 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.218 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.218 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.218 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.218 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.218 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.218 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.218 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.218 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.218 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.219 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.219 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.219 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.219 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.219 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.219 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.219 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.219 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.219 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.219 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.219 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.219 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.220 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.221 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.222 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.222 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.222 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.222 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.222 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.222 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.222 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.222 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.222 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.222 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.222 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.222 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.223 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.223 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.223 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.223 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.223 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.223 INFO:teuthology.orchestra.run.smithi132.stdout: "roption1": { 2024-04-23T18:46:09.223 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "roption1", 2024-04-23T18:46:09.223 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.223 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.223 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.223 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.223 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "roption2": { 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "roption2", 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "xyz", 2024-04-23T18:46:09.224 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption1": { 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption1", 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.225 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.226 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.226 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.226 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.226 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.226 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.226 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.226 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.226 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.226 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption2": { 2024-04-23T18:46:09.226 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption2", 2024-04-23T18:46:09.226 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.226 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption3": { 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption3", 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-04-23T18:46:09.227 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption4": { 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption4", 2024-04-23T18:46:09.228 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption5": { 2024-04-23T18:46:09.229 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption5", 2024-04-23T18:46:09.230 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.230 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.230 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.230 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.230 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.230 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.230 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.230 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.230 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.230 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.230 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption6": { 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption6", 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.231 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption7": { 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption7", 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "42", 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.232 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: "testkey": { 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testkey", 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.233 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: "testlkey": { 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testlkey", 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.234 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: "testnewline": { 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testnewline", 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snap_schedule", 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.236 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_m_granularity": { 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_m_granularity", 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.237 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.238 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.238 INFO:teuthology.orchestra.run.smithi132.stdout: "dump_on_update": { 2024-04-23T18:46:09.238 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "dump_on_update", 2024-04-23T18:46:09.238 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.238 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.238 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.238 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.238 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.238 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.238 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.238 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "dump database to debug log on update", 2024-04-23T18:46:09.238 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.239 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.240 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.241 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.242 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.242 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.242 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.242 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.242 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.242 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.242 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.242 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.242 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.242 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.242 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.242 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.243 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.244 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "stats", 2024-04-23T18:46:09.245 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.245 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.245 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.245 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.245 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.245 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.245 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.245 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.245 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.245 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.245 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.245 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.246 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.247 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.247 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.247 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.247 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.247 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.247 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.247 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.247 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.247 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.247 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.247 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.247 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.248 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.248 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.248 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.248 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.248 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.248 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.248 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.248 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.248 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.248 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.248 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.248 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.249 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.249 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.249 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.249 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.249 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.249 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.249 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.249 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.249 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.249 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.249 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.249 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.250 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.250 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.250 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.250 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.250 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.250 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.250 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.250 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.250 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.250 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.250 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.250 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "status", 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.251 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.252 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.252 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.252 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.252 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.252 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.252 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.252 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.252 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.252 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.252 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.252 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.253 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.253 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.253 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.253 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.253 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.253 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.253 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.253 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.253 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.253 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.253 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.255 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.255 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.255 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.255 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.255 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.255 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.255 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.255 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.255 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.255 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.255 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.255 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.256 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.257 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.257 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.257 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.257 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.257 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.257 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.257 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.257 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.257 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.257 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "telegraf", 2024-04-23T18:46:09.257 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.257 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "address": { 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "address", 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.258 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.259 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.259 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.259 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:09.259 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:09.259 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:09.259 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.259 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.259 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15", 2024-04-23T18:46:09.259 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.259 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.259 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.259 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.260 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.260 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.260 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.260 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.260 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.260 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.260 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.260 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.260 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.260 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.260 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.260 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.261 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.262 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.262 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.262 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.262 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.262 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.262 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.262 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.262 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.262 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.262 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.262 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.262 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.263 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.263 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.263 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.263 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.263 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.263 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.263 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.263 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.263 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.263 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.263 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.263 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.264 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.265 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.266 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.266 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "telemetry", 2024-04-23T18:46:09.266 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.266 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.266 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.266 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_basic": { 2024-04-23T18:46:09.266 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_basic", 2024-04-23T18:46:09.266 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.266 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.266 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.266 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.266 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.267 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.267 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.267 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T18:46:09.267 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.267 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.267 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.267 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.267 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_crash": { 2024-04-23T18:46:09.267 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_crash", 2024-04-23T18:46:09.267 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.267 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.267 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.268 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.268 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.268 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.268 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.268 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T18:46:09.268 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.268 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.268 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.268 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.268 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_device": { 2024-04-23T18:46:09.268 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_device", 2024-04-23T18:46:09.269 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.269 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.269 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.269 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.269 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.269 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.269 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.269 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T18:46:09.269 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.269 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.269 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.269 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.270 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_ident": { 2024-04-23T18:46:09.270 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_ident", 2024-04-23T18:46:09.270 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.270 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.270 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.270 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.270 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.270 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.270 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.270 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T18:46:09.270 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.271 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.271 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.271 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.271 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_perf": { 2024-04-23T18:46:09.271 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_perf", 2024-04-23T18:46:09.271 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.271 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.271 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.271 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.271 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.271 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.271 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: "contact": { 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "contact", 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.273 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.273 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.273 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.273 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.273 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.273 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.273 INFO:teuthology.orchestra.run.smithi132.stdout: "description": { 2024-04-23T18:46:09.273 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "description", 2024-04-23T18:46:09.273 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.273 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.273 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.273 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi132.stdout: "device_url": { 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_url", 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.274 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.275 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.275 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T18:46:09.275 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.275 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.275 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.275 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.275 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.275 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.275 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.275 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.275 INFO:teuthology.orchestra.run.smithi132.stdout: "enabled": { 2024-04-23T18:46:09.275 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enabled", 2024-04-23T18:46:09.276 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.276 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.276 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.276 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.276 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.276 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.276 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.276 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.276 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.276 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.276 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.276 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.277 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:09.277 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:09.277 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.277 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.277 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.277 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "24", 2024-04-23T18:46:09.277 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "8", 2024-04-23T18:46:09.277 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.277 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.277 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.277 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.277 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.278 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.278 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.278 INFO:teuthology.orchestra.run.smithi132.stdout: "last_opt_revision": { 2024-04-23T18:46:09.278 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "last_opt_revision", 2024-04-23T18:46:09.278 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.278 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.278 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.278 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1", 2024-04-23T18:46:09.278 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.278 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.278 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.278 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "leaderboard": { 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "leaderboard", 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.279 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.280 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.280 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.280 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.280 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.280 INFO:teuthology.orchestra.run.smithi132.stdout: "leaderboard_description": { 2024-04-23T18:46:09.280 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "leaderboard_description", 2024-04-23T18:46:09.280 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.280 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.280 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.280 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.280 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.280 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.281 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.281 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.281 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.281 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.281 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.281 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.281 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.281 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.281 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.281 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.281 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.281 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.282 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.283 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.284 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.284 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.284 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.284 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.284 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.284 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.284 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.284 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.284 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.284 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.284 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.284 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.285 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.285 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.285 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.285 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.285 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.285 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.285 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.285 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.285 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.285 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.285 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.285 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.286 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.286 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.286 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.286 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.286 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.286 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.286 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.286 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.286 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.286 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.286 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.286 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: "organization": { 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "organization", 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.287 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.288 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.288 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.288 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.288 INFO:teuthology.orchestra.run.smithi132.stdout: "proxy": { 2024-04-23T18:46:09.288 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "proxy", 2024-04-23T18:46:09.288 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.288 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.288 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.288 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.288 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.288 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.288 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.289 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.289 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.289 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.289 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.289 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.289 INFO:teuthology.orchestra.run.smithi132.stdout: "url": { 2024-04-23T18:46:09.289 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "url", 2024-04-23T18:46:09.289 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.289 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.289 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.289 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T18:46:09.289 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "test_orchestrator", 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.290 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.291 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.292 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.293 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.294 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.295 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.295 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.295 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.295 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.295 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.295 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.295 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.295 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.295 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.295 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.295 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.295 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "volumes", 2024-04-23T18:46:09.296 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.297 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.298 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.298 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.298 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.298 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.298 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.298 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.298 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.298 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.298 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.298 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.298 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.298 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.299 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.300 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.301 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.302 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.302 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.302 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.302 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.302 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.302 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.302 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.302 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.302 INFO:teuthology.orchestra.run.smithi132.stdout: "max_concurrent_clones": { 2024-04-23T18:46:09.302 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_concurrent_clones", 2024-04-23T18:46:09.302 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.302 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "4", 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: "periodic_async_work": { 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "periodic_async_work", 2024-04-23T18:46:09.303 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Periodically check for async work", 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: "snapshot_clone_delay": { 2024-04-23T18:46:09.304 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snapshot_clone_delay", 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.305 INFO:teuthology.orchestra.run.smithi132.stdout: "snapshot_clone_no_wait": { 2024-04-23T18:46:09.306 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T18:46:09.306 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.306 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.306 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.306 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-04-23T18:46:09.306 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.306 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.306 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.306 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T18:46:09.306 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.306 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix", 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: "discovery_interval": { 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "discovery_interval", 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-04-23T18:46:09.307 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.308 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.308 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "100", 2024-04-23T18:46:09.308 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.308 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.308 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.308 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.308 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.308 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.308 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.308 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.308 INFO:teuthology.orchestra.run.smithi132.stdout: "identifier": { 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "identifier", 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.309 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.310 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-04-23T18:46:09.310 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-04-23T18:46:09.310 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-04-23T18:46:09.310 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.310 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.310 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-04-23T18:46:09.310 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.310 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.310 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.310 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.310 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.310 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.311 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.311 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.311 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-04-23T18:46:09.311 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-04-23T18:46:09.311 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.311 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.311 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.311 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.311 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.311 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.311 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.311 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-04-23T18:46:09.312 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-04-23T18:46:09.313 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.313 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.313 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.313 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.313 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.313 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.313 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.313 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.313 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.313 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.314 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.314 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.314 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-04-23T18:46:09.314 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-04-23T18:46:09.314 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.314 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.314 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.314 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-04-23T18:46:09.314 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.314 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.314 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-04-23T18:46:09.315 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-04-23T18:46:09.316 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-04-23T18:46:09.316 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.316 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-04-23T18:46:09.316 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-04-23T18:46:09.316 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.316 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.316 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.316 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.316 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.316 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.316 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_host": { 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_host", 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.317 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.318 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.318 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_port": { 2024-04-23T18:46:09.318 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_port", 2024-04-23T18:46:09.318 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-04-23T18:46:09.318 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.318 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.318 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10051", 2024-04-23T18:46:09.318 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.318 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.318 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.318 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.319 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.319 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.319 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.319 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.319 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_sender": { 2024-04-23T18:46:09.319 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_sender", 2024-04-23T18:46:09.319 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-04-23T18:46:09.319 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-04-23T18:46:09.319 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-04-23T18:46:09.319 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: "services": { 2024-04-23T18:46:09.320 INFO:teuthology.orchestra.run.smithi132.stdout: "dashboard": "https://172.21.15.132:8443/" 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: "always_on_modules": { 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: "octopus": [ 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: "balancer", 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: "crash", 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: "devicehealth", 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator", 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_autoscaler", 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: "progress", 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_support", 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: "status", 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: "telemetry", 2024-04-23T18:46:09.321 INFO:teuthology.orchestra.run.smithi132.stdout: "volumes" 2024-04-23T18:46:09.322 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.322 INFO:teuthology.orchestra.run.smithi132.stdout: "pacific": [ 2024-04-23T18:46:09.322 INFO:teuthology.orchestra.run.smithi132.stdout: "balancer", 2024-04-23T18:46:09.322 INFO:teuthology.orchestra.run.smithi132.stdout: "crash", 2024-04-23T18:46:09.322 INFO:teuthology.orchestra.run.smithi132.stdout: "devicehealth", 2024-04-23T18:46:09.322 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator", 2024-04-23T18:46:09.322 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_autoscaler", 2024-04-23T18:46:09.322 INFO:teuthology.orchestra.run.smithi132.stdout: "progress", 2024-04-23T18:46:09.322 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_support", 2024-04-23T18:46:09.322 INFO:teuthology.orchestra.run.smithi132.stdout: "status", 2024-04-23T18:46:09.322 INFO:teuthology.orchestra.run.smithi132.stdout: "telemetry", 2024-04-23T18:46:09.322 INFO:teuthology.orchestra.run.smithi132.stdout: "volumes" 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: "quincy": [ 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: "balancer", 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: "crash", 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: "devicehealth", 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator", 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_autoscaler", 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: "progress", 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_support", 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: "status", 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: "telemetry", 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: "volumes" 2024-04-23T18:46:09.323 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "reef": [ 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "balancer", 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "crash", 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "devicehealth", 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator", 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_autoscaler", 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "progress", 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_support", 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "status", 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "telemetry", 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "volumes" 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "squid": [ 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "balancer", 2024-04-23T18:46:09.324 INFO:teuthology.orchestra.run.smithi132.stdout: "crash", 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: "devicehealth", 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator", 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_autoscaler", 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: "progress", 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_support", 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: "status", 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: "telemetry", 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: "volumes" 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: "last_failure_osd_epoch": 91, 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: "active_clients": [ 2024-04-23T18:46:09.325 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "devicehealth", 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:0", 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 1213114499 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "libcephsqlite", 2024-04-23T18:46:09.326 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:0", 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 910133454 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_support", 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-04-23T18:46:09.327 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:0", 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 214525185 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snap_schedule", 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:0", 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 3727009134 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.328 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-04-23T18:46:09.329 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.329 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.329 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snap_schedule", 2024-04-23T18:46:09.329 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-04-23T18:46:09.329 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.329 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-04-23T18:46:09.329 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:0", 2024-04-23T18:46:09.329 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 3382455757 2024-04-23T18:46:09.329 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.329 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-04-23T18:46:09.329 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:09.329 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.330 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "volumes", 2024-04-23T18:46:09.330 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-04-23T18:46:09.330 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-04-23T18:46:09.330 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-04-23T18:46:09.330 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:0", 2024-04-23T18:46:09.330 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 1697565249 2024-04-23T18:46:09.330 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.330 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-04-23T18:46:09.330 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:09.330 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-04-23T18:46:09.330 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:46:09.332 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:07 smithi157 ceph-mon[32580]: from='client.15912 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:46:09.332 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:07 smithi157 ceph-mon[32580]: from='client.15918 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:46:09.332 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:07 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1175205959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:46:09.332 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:07 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2160803175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:46:09.332 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:07 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3328472058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:46:09.332 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:08 smithi157 ceph-mon[32580]: pgmap v311: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T18:46:09.333 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:08 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/551979836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T18:46:09.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:07 smithi148 ceph-mon[32311]: from='client.15912 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:46:09.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:07 smithi148 ceph-mon[32311]: from='client.15918 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:46:09.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:07 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1175205959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:46:09.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:07 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2160803175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:46:09.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:07 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3328472058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:46:09.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:08 smithi148 ceph-mon[32311]: pgmap v311: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T18:46:09.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:08 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/551979836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T18:46:09.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.920+0000 7f9b4e272640 1 -- 172.21.15.132:0/551979836 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9b300a3da0 msgr2=0x7f9b300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:09.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.920+0000 7f9b4e272640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9b300a3da0 0x7f9b300a6260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800a810 tx=0x7f9b38008040 comp rx=0 tx=0).stop 2024-04-23T18:46:09.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.920+0000 7f9b4e272640 1 -- 172.21.15.132:0/551979836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b48103b00 msgr2=0x7f9b48198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:09.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.920+0000 7f9b4e272640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b48103b00 0x7f9b48198c50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00c890 tx=0x7f9b3c00cd50 comp rx=0 tx=0).stop 2024-04-23T18:46:09.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.920+0000 7f9b4e272640 1 -- 172.21.15.132:0/551979836 shutdown_connections 2024-04-23T18:46:09.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.920+0000 7f9b4e272640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f9b300a3da0 0x7f9b300a6260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:09.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.920+0000 7f9b4e272640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b4810c520 0x7f9b4819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:09.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.920+0000 7f9b4e272640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b48103b00 0x7f9b48198c50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:09.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.920+0000 7f9b4e272640 1 --2- 172.21.15.132:0/551979836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b481031e0 0x7f9b48198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:09.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.920+0000 7f9b4e272640 1 -- 172.21.15.132:0/551979836 >> 172.21.15.132:0/551979836 conn(0x7f9b48077f50 msgr2=0x7f9b480fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:09.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.920+0000 7f9b4e272640 1 -- 172.21.15.132:0/551979836 shutdown_connections 2024-04-23T18:46:09.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:07.920+0000 7f9b4e272640 1 -- 172.21.15.132:0/551979836 wait complete. 2024-04-23T18:46:09.341 DEBUG:tasks.check_counter:Getting stats from a 2024-04-23T18:46:09.341 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-23T18:46:09.341 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:08 smithi132 ceph-mon[29178]: pgmap v311: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T18:46:09.341 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:08 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/551979836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T18:46:09.592 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:46:11.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:10 smithi148 ceph-mon[32311]: pgmap v312: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 33 op/s 2024-04-23T18:46:11.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:10 smithi148 ceph-mon[32311]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T18:46:11.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:10 smithi157 ceph-mon[32580]: pgmap v312: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 33 op/s 2024-04-23T18:46:11.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:10 smithi157 ceph-mon[32580]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T18:46:11.044 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:10 smithi132 ceph-mon[29178]: pgmap v312: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 33 op/s 2024-04-23T18:46:11.044 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:10 smithi132 ceph-mon[29178]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T18:46:11.464 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:46:11.465 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:46:11.465 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 959, 2024-04-23T18:46:11.465 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 941, 2024-04-23T18:46:11.465 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1889104, 2024-04-23T18:46:11.465 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 355450, 2024-04-23T18:46:11.465 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 5, 2024-04-23T18:46:11.465 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 1, 2024-04-23T18:46:11.465 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.132390071, 2024-04-23T18:46:11.465 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.043366580, 2024-04-23T18:46:11.465 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.057006439, 2024-04-23T18:46:11.465 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.032184511, 2024-04-23T18:46:11.465 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 941, 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.050902896, 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000054094 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 1646176, 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 125536 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 5, 2024-04-23T18:46:11.466 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 155, 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1710, 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 433534, 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 7, 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 5, 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.012818783, 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.007876062, 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.003053864, 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000061335, 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 152, 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.009354437, 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000061542 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.467 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 432512 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 1377, 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 1379, 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 353250, 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 437887, 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 5, 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 4, 2024-04-23T18:46:11.468 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.110798555, 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.030238755, 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.028541874, 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.043222382, 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1377, 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.060287658, 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000043781 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.469 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 1, 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-MDSRank": { 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3451, 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.107878709, 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000031260 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.470 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-PurgeQueue": { 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: "mds": { 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: "request": 0, 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: "reply": 0, 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: "reply_latency": { 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.471 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_reply": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "forward": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_complete": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_keys": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_commit": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_split": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_merge": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_top": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_bottom": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pin_tail": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pinned": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_expired": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_with_caps": 0, 2024-04-23T18:46:11.472 INFO:teuthology.orchestra.run.smithi132.stdout: "caps": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "subtrees": 1, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_hit": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_forward": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_discover": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_remote_ino": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_lock": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "load_cent": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "q": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "exported": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "exported_inodes": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "imported": 0, 2024-04-23T18:46:11.473 INFO:teuthology.orchestra.run.smithi132.stdout: "imported_inodes": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_dir_fetch": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_peer_discover": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rfiles": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rbytes": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rsnaps": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_set_tag": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_file_inodes": 0, 2024-04-23T18:46:11.474 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_cap_release": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "process_request_cap_release": 0 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_cache": { 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update_receipt": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_try_discover": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_send_discover": 0, 2024-04-23T18:46:11.475 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_handle_discover": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_delayed": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_created": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_enqueued": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_reintegrated": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_migrated": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_processing": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_started": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_completed": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:46:11.476 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_exportdir": 0, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_flush": 0, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragstats": 0, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_inodestats": 0 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_log": { 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "evlrg": 0, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "evadd": 0, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "evex": 0, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "evtrm": 0, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "ev": 10, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "evexg": 0, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "evexd": 0, 2024-04-23T18:46:11.477 INFO:teuthology.orchestra.run.smithi132.stdout: "segadd": 0, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "segex": 0, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "segtrm": 0, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "seg": 2, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "segmjr": 2, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "segexg": 0, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "segexd": 0, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "expos": 4194304, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "wrpos": 4197187, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "rdpos": 4197046, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "jlat": { 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.478 INFO:teuthology.orchestra.run.smithi132.stdout: "replayed": 10 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_mem": { 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "ino": 3, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "ino+": 3, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "ino-": 0, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "dir": 1, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "dir+": 1, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "dir-": 0, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "dn": 0, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "dn+": 0, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "dn-": 0, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "cap": 0, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "cap+": 0, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "cap-": 0, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "rss": 53552, 2024-04-23T18:46:11.479 INFO:teuthology.orchestra.run.smithi132.stdout: "heap": 215416 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_server": { 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_client_request": 0, 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_server_request": 0, 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_request": 0, 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_session": 0, 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_peer_request": 0, 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "req_create_latency": { 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getattr_latency": { 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.480 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getfilelock_latency": { 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: "req_link_latency": { 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookup_latency": { 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.481 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookuphash_latency": { 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupino_latency": { 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupname_latency": { 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.482 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupparent_latency": { 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupsnap_latency": { 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lssnap_latency": { 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.483 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mkdir_latency": { 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mknod_latency": { 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mksnap_latency": { 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.484 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "req_open_latency": { 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "req_readdir_latency": { 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rename_latency": { 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.485 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: "req_renamesnap_latency": { 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: "req_snapdiff_latency": { 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmdir_latency": { 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.486 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmsnap_latency": { 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmxattr_latency": { 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setattr_latency": { 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.487 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setdirlayout_latency": { 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setfilelock_latency": { 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setlayout_latency": { 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.488 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.489 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.489 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setxattr_latency": { 2024-04-23T18:46:11.489 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.489 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.489 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.489 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.489 INFO:teuthology.orchestra.run.smithi132.stdout: "req_symlink_latency": { 2024-04-23T18:46:11.489 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.489 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.490 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.490 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.490 INFO:teuthology.orchestra.run.smithi132.stdout: "req_unlink_latency": { 2024-04-23T18:46:11.490 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.490 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.490 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.490 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.490 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:46:11.490 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:46:11.490 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getvxattr_latency": { 2024-04-23T18:46:11.491 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.491 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.491 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.491 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:11.491 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.491 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_sessions": { 2024-04-23T18:46:11.491 INFO:teuthology.orchestra.run.smithi132.stdout: "session_count": 2, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "session_add": 5, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "session_remove": 3, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_open": 4, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_stale": 0, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "total_load": 0, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "average_load": 0, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "avg_session_uptime": 3056, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "mdthresh_evicted": 0 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:46:11.492 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:46:11.493 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:46:11.493 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:46:11.493 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:46:11.493 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:46:11.494 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:46:11.494 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:46:11.494 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:46:11.494 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:46:11.494 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:46:11.494 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:46:11.494 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2024-04-23T18:46:11.494 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:46:11.494 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2024-04-23T18:46:11.494 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:46:11.494 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:46:11.494 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:46:11.495 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 197530, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 57, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 8232, 2024-04-23T18:46:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 172, 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 8512, 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 13, 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "objecter": { 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "op_active": 0, 2024-04-23T18:46:11.497 INFO:teuthology.orchestra.run.smithi132.stdout: "op_laggy": 0, 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send": 2103, 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send_bytes": 21, 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: "op_resend": 3, 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: "op_reply": 2103, 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: "op_latency": { 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2103, 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1.316964442, 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000626231 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: "op_inflight": 0, 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: "oplen_avg": { 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2103, 2024-04-23T18:46:11.498 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 2104 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "op": 2103, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "op_r": 2103, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "op_w": 0, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "op_rmw": 0, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "op_pg": 0, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_stat": 1369, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_create": 0, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_read": 732, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_write": 0, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writefull": 0, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writesame": 0, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_append": 0, 2024-04-23T18:46:11.499 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_zero": 0, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_truncate": 0, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_delete": 0, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_mapext": 0, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_sparse_read": 0, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_clonerange": 0, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_getxattr": 1, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_setxattr": 0, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_rmxattr": 0, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_call": 0, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_watch": 0, 2024-04-23T18:46:11.500 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_notify": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_other": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_active": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_send": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_resend": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_ping": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_active": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_send": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_resend": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_active": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_send": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_resend": 0, 2024-04-23T18:46:11.501 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_active": 0, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_send": 0, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_resend": 0, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "command_active": 0, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "command_send": 0, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "command_resend": 0, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "map_epoch": 98, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "map_full": 0, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "map_inc": 12, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_sessions": 6, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_open": 6, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_close": 0, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_laggy": 0, 2024-04-23T18:46:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_wr": 0, 2024-04-23T18:46:11.503 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_rd": 2, 2024-04-23T18:46:11.503 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_del": 0 2024-04-23T18:46:11.503 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.503 INFO:teuthology.orchestra.run.smithi132.stdout: "oft": { 2024-04-23T18:46:11.503 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_objs": 0, 2024-04-23T18:46:11.503 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:46:11.503 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_updates": 0, 2024-04-23T18:46:11.503 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_removes": 0 2024-04-23T18:46:11.503 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.503 INFO:teuthology.orchestra.run.smithi132.stdout: "osd-slow-ops": { 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_ops_count": 0 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "purge_queue": { 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops": 0, 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing": 0, 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_high_water": 0, 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed_ops": 0, 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed": 0, 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_item_in_journal": 0 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-04-23T18:46:11.504 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 2337, 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 2018896, 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 2337, 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 2337, 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 2018896, 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:11.505 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.506 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_bytes": { 2024-04-23T18:46:11.506 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-04-23T18:46:11.506 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-04-23T18:46:11.506 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-04-23T18:46:11.506 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-04-23T18:46:11.506 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-04-23T18:46:11.506 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:11.506 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:11.506 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 2103, 2024-04-23T18:46:11.506 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 2888, 2024-04-23T18:46:11.506 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 2103, 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 2888, 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_ops": { 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 1024, 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:11.507 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 2103, 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 2103, 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 2103, 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 2103, 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2024-04-23T18:46:11.508 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 0, 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 0, 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:11.509 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle-0x556e34d266e0": { 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 0, 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 0, 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:11.510 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:11.511 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:11.511 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:11.511 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:11.511 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:46:12.167 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-23T18:46:12.167 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-23T18:46:12.167 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-23T18:46:12.167 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:46:12.167 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:46:12.167 DEBUG:tasks.check_counter:Getting stats from d 2024-04-23T18:46:12.168 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-23T18:46:12.422 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:46:12.718 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:12 smithi132 ceph-mon[29178]: pgmap v314: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 34 op/s 2024-04-23T18:46:13.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:12 smithi148 ceph-mon[32311]: pgmap v314: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 34 op/s 2024-04-23T18:46:13.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:12 smithi157 ceph-mon[32580]: pgmap v314: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 34 op/s 2024-04-23T18:46:14.128 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:46:14.128 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:46:14.128 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 4, 2024-04-23T18:46:14.128 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 4, 2024-04-23T18:46:14.128 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 2992, 2024-04-23T18:46:14.128 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 448, 2024-04-23T18:46:14.128 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 4, 2024-04-23T18:46:14.128 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 0, 2024-04-23T18:46:14.128 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.001509098, 2024-04-23T18:46:14.128 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.000040862, 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.000547194, 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3, 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000709479, 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000236493 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T18:46:14.129 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2, 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 157, 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 320, 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 183744, 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 5, 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.013536923, 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.009314159, 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.001823808, 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:14.130 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 154, 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.011604430, 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000075353 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 183744 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 216, 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 205, 2024-04-23T18:46:14.131 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1588240, 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 125984, 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 2, 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 1, 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.041751632, 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.013889881, 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.032891894, 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.004152944, 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 204, 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.009545069, 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000046789 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:14.132 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 1588240, 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 125984 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 0, 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:46:14.133 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2024-04-23T18:46:14.134 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2024-04-23T18:46:14.135 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 149057, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 32, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 0, 2024-04-23T18:46:14.136 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 0, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 928, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 2, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-04-23T18:46:14.137 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 218, 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1557789, 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 218, 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 218, 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1557789, 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:14.138 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:46:14.823 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:14 smithi132 ceph-mon[29178]: pgmap v315: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 34 op/s 2024-04-23T18:46:14.852 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-23T18:46:14.852 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-23T18:46:14.852 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-23T18:46:14.852 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-23T18:46:14.852 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-23T18:46:14.852 DEBUG:tasks.check_counter:Getting stats from g 2024-04-23T18:46:14.853 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-23T18:46:15.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:14 smithi148 ceph-mon[32311]: pgmap v315: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 34 op/s 2024-04-23T18:46:15.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:14 smithi157 ceph-mon[32580]: pgmap v315: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 34 op/s 2024-04-23T18:46:15.094 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:46:15.860 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:15 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:46:16.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:15 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:46:16.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:15 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:46:16.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:16 smithi148 ceph-mon[32311]: pgmap v316: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-23T18:46:16.789 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:16 smithi132 ceph-mon[29178]: pgmap v316: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-23T18:46:16.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:16 smithi157 ceph-mon[32580]: pgmap v316: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-23T18:46:16.789 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:46:16.789 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:46:16.789 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 730, 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 732, 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 245225, 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 232141, 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 6, 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 2, 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.085509362, 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.036301238, 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.017463989, 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.022440091, 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 730, 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.041706254, 2024-04-23T18:46:16.790 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000057131 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 1586, 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 1570, 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1990425, 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 556683, 2024-04-23T18:46:16.791 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 4, 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.135758866, 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.039040652, 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.061612254, 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.039224884, 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1569, 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.053526956, 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000034115 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.792 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 1644000, 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 127008 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 9, 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 159, 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 4660, 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 437509, 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 7, 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 5, 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.014070803, 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.008677840, 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.003150189, 2024-04-23T18:46:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000053231, 2024-04-23T18:46:16.794 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:16.794 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 157, 2024-04-23T18:46:16.794 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.013474365, 2024-04-23T18:46:16.794 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000085823 2024-04-23T18:46:16.794 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.794 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:16.794 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.794 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.794 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.794 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.794 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-23T18:46:16.794 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 436544 2024-04-23T18:46:16.795 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.795 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2024-04-23T18:46:16.795 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 1, 2024-04-23T18:46:16.795 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2024-04-23T18:46:16.795 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.795 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-MDSRank": { 2024-04-23T18:46:16.795 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2024-04-23T18:46:16.795 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2024-04-23T18:46:16.795 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3502, 2024-04-23T18:46:16.795 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.086789317, 2024-04-23T18:46:16.795 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000024782 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-PurgeQueue": { 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: "mds": { 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: "request": 0, 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: "reply": 0, 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: "reply_latency": { 2024-04-23T18:46:16.796 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_reply": 0, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "forward": 0, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_complete": 0, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_keys": 0, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_commit": 0, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_split": 0, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_merge": 0, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes": 0, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_top": 0, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_bottom": 0, 2024-04-23T18:46:16.797 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pin_tail": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pinned": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_expired": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_with_caps": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "caps": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "subtrees": 1, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_hit": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_forward": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_discover": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_remote_ino": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_lock": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "load_cent": 0, 2024-04-23T18:46:16.798 INFO:teuthology.orchestra.run.smithi132.stdout: "q": 0, 2024-04-23T18:46:16.799 INFO:teuthology.orchestra.run.smithi132.stdout: "exported": 0, 2024-04-23T18:46:16.799 INFO:teuthology.orchestra.run.smithi132.stdout: "exported_inodes": 0, 2024-04-23T18:46:16.799 INFO:teuthology.orchestra.run.smithi132.stdout: "imported": 0, 2024-04-23T18:46:16.799 INFO:teuthology.orchestra.run.smithi132.stdout: "imported_inodes": 0, 2024-04-23T18:46:16.799 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_dir_fetch": 0, 2024-04-23T18:46:16.799 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:46:16.799 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_peer_discover": 0, 2024-04-23T18:46:16.800 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rfiles": 0, 2024-04-23T18:46:16.800 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rbytes": 0, 2024-04-23T18:46:16.800 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rsnaps": 0, 2024-04-23T18:46:16.800 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:46:16.800 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_set_tag": 0, 2024-04-23T18:46:16.800 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:46:16.801 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:46:16.801 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:46:16.801 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:46:16.801 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:46:16.801 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_file_inodes": 0, 2024-04-23T18:46:16.801 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:46:16.801 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:46:16.802 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:46:16.802 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:46:16.802 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:46:16.802 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:46:16.802 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps": 0, 2024-04-23T18:46:16.802 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:46:16.803 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_cap_release": 0, 2024-04-23T18:46:16.803 INFO:teuthology.orchestra.run.smithi132.stdout: "process_request_cap_release": 0 2024-04-23T18:46:16.803 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.803 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_cache": { 2024-04-23T18:46:16.803 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update": 0, 2024-04-23T18:46:16.803 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update_receipt": 0, 2024-04-23T18:46:16.804 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_try_discover": 0, 2024-04-23T18:46:16.804 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_send_discover": 0, 2024-04-23T18:46:16.804 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_handle_discover": 0, 2024-04-23T18:46:16.804 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays": 0, 2024-04-23T18:46:16.804 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_delayed": 0, 2024-04-23T18:46:16.804 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:46:16.804 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_created": 0, 2024-04-23T18:46:16.804 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_enqueued": 0, 2024-04-23T18:46:16.805 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_reintegrated": 0, 2024-04-23T18:46:16.805 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_migrated": 0, 2024-04-23T18:46:16.805 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_processing": 0, 2024-04-23T18:46:16.805 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:46:16.805 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:46:16.805 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_started": 0, 2024-04-23T18:46:16.805 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_completed": 0, 2024-04-23T18:46:16.806 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:46:16.806 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:46:16.806 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:46:16.806 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_exportdir": 0, 2024-04-23T18:46:16.806 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_flush": 0, 2024-04-23T18:46:16.806 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:46:16.806 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragstats": 0, 2024-04-23T18:46:16.806 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_inodestats": 0 2024-04-23T18:46:16.807 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.807 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_log": { 2024-04-23T18:46:16.807 INFO:teuthology.orchestra.run.smithi132.stdout: "evlrg": 0, 2024-04-23T18:46:16.807 INFO:teuthology.orchestra.run.smithi132.stdout: "evadd": 0, 2024-04-23T18:46:16.807 INFO:teuthology.orchestra.run.smithi132.stdout: "evex": 0, 2024-04-23T18:46:16.807 INFO:teuthology.orchestra.run.smithi132.stdout: "evtrm": 0, 2024-04-23T18:46:16.807 INFO:teuthology.orchestra.run.smithi132.stdout: "ev": 8, 2024-04-23T18:46:16.808 INFO:teuthology.orchestra.run.smithi132.stdout: "evexg": 0, 2024-04-23T18:46:16.808 INFO:teuthology.orchestra.run.smithi132.stdout: "evexd": 0, 2024-04-23T18:46:16.808 INFO:teuthology.orchestra.run.smithi132.stdout: "segadd": 0, 2024-04-23T18:46:16.809 INFO:teuthology.orchestra.run.smithi132.stdout: "segex": 0, 2024-04-23T18:46:16.809 INFO:teuthology.orchestra.run.smithi132.stdout: "segtrm": 0, 2024-04-23T18:46:16.809 INFO:teuthology.orchestra.run.smithi132.stdout: "seg": 2, 2024-04-23T18:46:16.809 INFO:teuthology.orchestra.run.smithi132.stdout: "segmjr": 2, 2024-04-23T18:46:16.809 INFO:teuthology.orchestra.run.smithi132.stdout: "segexg": 0, 2024-04-23T18:46:16.809 INFO:teuthology.orchestra.run.smithi132.stdout: "segexd": 0, 2024-04-23T18:46:16.810 INFO:teuthology.orchestra.run.smithi132.stdout: "expos": 4194304, 2024-04-23T18:46:16.810 INFO:teuthology.orchestra.run.smithi132.stdout: "wrpos": 4196655, 2024-04-23T18:46:16.810 INFO:teuthology.orchestra.run.smithi132.stdout: "rdpos": 4196514, 2024-04-23T18:46:16.810 INFO:teuthology.orchestra.run.smithi132.stdout: "jlat": { 2024-04-23T18:46:16.810 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.810 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.810 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.810 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.810 INFO:teuthology.orchestra.run.smithi132.stdout: "replayed": 8 2024-04-23T18:46:16.810 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_mem": { 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "ino": 3, 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "ino+": 3, 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "ino-": 0, 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "dir": 1, 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "dir+": 1, 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "dir-": 0, 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "dn": 0, 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "dn+": 0, 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "dn-": 0, 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "cap": 0, 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "cap+": 0, 2024-04-23T18:46:16.811 INFO:teuthology.orchestra.run.smithi132.stdout: "cap-": 0, 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "rss": 50024, 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "heap": 215416 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_server": { 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_client_request": 0, 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_server_request": 0, 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_request": 0, 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_session": 0, 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_peer_request": 0, 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "req_create_latency": { 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getattr_latency": { 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.812 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getfilelock_latency": { 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: "req_link_latency": { 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookup_latency": { 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookuphash_latency": { 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupino_latency": { 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupname_latency": { 2024-04-23T18:46:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupparent_latency": { 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupsnap_latency": { 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lssnap_latency": { 2024-04-23T18:46:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.816 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.816 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.816 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.816 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mkdir_latency": { 2024-04-23T18:46:16.816 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.816 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mknod_latency": { 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mksnap_latency": { 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "req_open_latency": { 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.817 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: "req_readdir_latency": { 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rename_latency": { 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: "req_renamesnap_latency": { 2024-04-23T18:46:16.818 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.819 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.819 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.819 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.819 INFO:teuthology.orchestra.run.smithi132.stdout: "req_snapdiff_latency": { 2024-04-23T18:46:16.819 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.819 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.819 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.819 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.820 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmdir_latency": { 2024-04-23T18:46:16.820 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.820 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.820 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.820 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.820 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmsnap_latency": { 2024-04-23T18:46:16.820 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.820 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.820 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.820 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.820 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmxattr_latency": { 2024-04-23T18:46:16.821 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.821 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.821 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.821 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.821 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setattr_latency": { 2024-04-23T18:46:16.821 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.821 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.821 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.821 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.821 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setdirlayout_latency": { 2024-04-23T18:46:16.822 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.822 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.822 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.822 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.822 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setfilelock_latency": { 2024-04-23T18:46:16.823 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.823 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.823 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.823 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.823 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setlayout_latency": { 2024-04-23T18:46:16.823 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.824 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.824 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.824 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.824 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setxattr_latency": { 2024-04-23T18:46:16.824 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.824 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.825 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.825 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.825 INFO:teuthology.orchestra.run.smithi132.stdout: "req_symlink_latency": { 2024-04-23T18:46:16.825 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.825 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.825 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.825 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.825 INFO:teuthology.orchestra.run.smithi132.stdout: "req_unlink_latency": { 2024-04-23T18:46:16.825 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.826 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.826 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.826 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.826 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:46:16.826 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:46:16.826 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getvxattr_latency": { 2024-04-23T18:46:16.826 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.826 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.826 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.826 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:16.826 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.827 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_sessions": { 2024-04-23T18:46:16.827 INFO:teuthology.orchestra.run.smithi132.stdout: "session_count": 2, 2024-04-23T18:46:16.827 INFO:teuthology.orchestra.run.smithi132.stdout: "session_add": 4, 2024-04-23T18:46:16.827 INFO:teuthology.orchestra.run.smithi132.stdout: "session_remove": 2, 2024-04-23T18:46:16.827 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_open": 4, 2024-04-23T18:46:16.827 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_stale": 0, 2024-04-23T18:46:16.827 INFO:teuthology.orchestra.run.smithi132.stdout: "total_load": 0, 2024-04-23T18:46:16.827 INFO:teuthology.orchestra.run.smithi132.stdout: "average_load": 0, 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "avg_session_uptime": 3064, 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "mdthresh_evicted": 0 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:46:16.828 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:46:16.829 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:46:16.830 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2024-04-23T18:46:16.830 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:46:16.830 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:46:16.830 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:46:16.830 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2024-04-23T18:46:16.830 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2024-04-23T18:46:16.830 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2024-04-23T18:46:16.830 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:46:16.830 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 197451, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 55, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 8352, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 175, 2024-04-23T18:46:16.831 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 8512, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 13, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "objecter": { 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "op_active": 0, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "op_laggy": 0, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send": 2084, 2024-04-23T18:46:16.832 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send_bytes": 21, 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "op_resend": 3, 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "op_reply": 2084, 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "op_latency": { 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2084, 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.930974843, 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000446724 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "op_inflight": 0, 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "oplen_avg": { 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2084, 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 2085 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "op": 2084, 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "op_r": 2084, 2024-04-23T18:46:16.833 INFO:teuthology.orchestra.run.smithi132.stdout: "op_w": 0, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "op_rmw": 0, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "op_pg": 0, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_stat": 1344, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_create": 0, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_read": 738, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_write": 0, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writefull": 0, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writesame": 0, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_append": 0, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_zero": 0, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_truncate": 0, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_delete": 0, 2024-04-23T18:46:16.834 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_mapext": 0, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_sparse_read": 0, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_clonerange": 0, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_getxattr": 1, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_setxattr": 0, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_rmxattr": 0, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_call": 0, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_watch": 0, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_notify": 0, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls": 0, 2024-04-23T18:46:16.835 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:46:16.836 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_other": 0, 2024-04-23T18:46:16.836 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_active": 0, 2024-04-23T18:46:16.836 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_send": 0, 2024-04-23T18:46:16.836 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_resend": 0, 2024-04-23T18:46:16.836 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_ping": 0, 2024-04-23T18:46:16.836 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_active": 0, 2024-04-23T18:46:16.836 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_send": 0, 2024-04-23T18:46:16.836 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_resend": 0, 2024-04-23T18:46:16.836 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_active": 0, 2024-04-23T18:46:16.836 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_send": 0, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_resend": 0, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_active": 0, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_send": 0, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_resend": 0, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "command_active": 0, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "command_send": 0, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "command_resend": 0, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "map_epoch": 98, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "map_full": 0, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "map_inc": 12, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_sessions": 6, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_open": 6, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_close": 0, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_laggy": 0, 2024-04-23T18:46:16.837 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_wr": 0, 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_rd": 2, 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_del": 0 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: "oft": { 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_objs": 0, 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_updates": 0, 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_removes": 0 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: "osd-slow-ops": { 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_ops_count": 0 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.838 INFO:teuthology.orchestra.run.smithi132.stdout: "purge_queue": { 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops": 0, 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing": 0, 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_high_water": 0, 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed_ops": 0, 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed": 0, 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_item_in_journal": 0 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-04-23T18:46:16.839 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 2321, 2024-04-23T18:46:16.840 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 2016426, 2024-04-23T18:46:16.840 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:16.840 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 2321, 2024-04-23T18:46:16.840 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-04-23T18:46:16.840 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-04-23T18:46:16.840 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 2321, 2024-04-23T18:46:16.840 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 2016426, 2024-04-23T18:46:16.840 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-04-23T18:46:16.840 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.840 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.840 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_bytes": { 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 2084, 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 2356, 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 2084, 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 2356, 2024-04-23T18:46:16.841 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-04-23T18:46:16.842 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.842 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.842 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.842 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:16.842 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.842 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_ops": { 2024-04-23T18:46:16.842 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-04-23T18:46:16.842 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 1024, 2024-04-23T18:46:16.842 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-04-23T18:46:16.842 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-04-23T18:46:16.842 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 2084, 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 2084, 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 2084, 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 2084, 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-04-23T18:46:16.843 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 0, 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 0, 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.844 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle-0x55a136bf06e0": { 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 0, 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 0, 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-04-23T18:46:16.845 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:16.846 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:16.846 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:16.846 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:16.846 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:16.846 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:46:17.554 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-23T18:46:17.554 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-23T18:46:17.554 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-23T18:46:17.554 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:46:17.554 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:46:17.554 DEBUG:tasks.check_counter:Getting stats from j 2024-04-23T18:46:17.555 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-23T18:46:18.331 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:46:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: pgmap v317: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-23T18:46:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:18 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: pgmap v317: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-23T18:46:18.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:18.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:18.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:18.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:18.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:18 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: pgmap v317: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-23T18:46:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:18.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:18 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:19.666 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:19 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-23T18:46:19.666 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:19 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-23T18:46:19.666 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:19 smithi132 ceph-mon[29178]: Unable to set osd_memory_target on smithi148 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:46:19.666 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:19 smithi132 ceph-mon[29178]: Unable to set osd_memory_target on smithi157 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:46:19.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:19 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-23T18:46:19.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:19 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-23T18:46:19.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:19 smithi148 ceph-mon[32311]: Unable to set osd_memory_target on smithi148 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:46:19.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:19 smithi148 ceph-mon[32311]: Unable to set osd_memory_target on smithi157 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:46:19.788 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:19 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-04-23T18:46:19.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:19 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-04-23T18:46:19.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:19 smithi157 ceph-mon[32580]: Unable to set osd_memory_target on smithi148 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:46:19.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:19 smithi157 ceph-mon[32580]: Unable to set osd_memory_target on smithi157 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-23T18:46:20.095 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-23T18:46:20.095 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 0, 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 2, 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 0, 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 0, 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 4, 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 0, 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.001101960, 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.000000000, 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.000286436, 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:20.096 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2, 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 159, 2024-04-23T18:46:20.097 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 320, 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 184096, 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 5, 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.015018777, 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.010325005, 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.002251936, 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 156, 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.011854244, 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000075988 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:20.098 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 184096 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 222, 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 209, 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1591776, 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 127712, 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 2, 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 1, 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.047377005, 2024-04-23T18:46:20.099 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.013842700, 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.038929581, 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.004452876, 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 209, 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.011770431, 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000056317 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 1591776, 2024-04-23T18:46:20.100 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 127712 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 0, 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:46:20.101 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:46:20.102 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 149017, 2024-04-23T18:46:20.103 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 31, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2024-04-23T18:46:20.104 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 928, 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 2, 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 220, 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1558049, 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:20.105 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 220, 2024-04-23T18:46:20.106 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-04-23T18:46:20.106 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-04-23T18:46:20.106 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 220, 2024-04-23T18:46:20.106 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1558049, 2024-04-23T18:46:20.106 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-04-23T18:46:20.106 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-04-23T18:46:20.106 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-04-23T18:46:20.106 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-04-23T18:46:20.106 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:20.106 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-23T18:46:20.106 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-23T18:46:20.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:20 smithi148 ceph-mon[32311]: pgmap v318: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:46:20.787 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:20 smithi132 ceph-mon[29178]: pgmap v318: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:46:20.788 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:20 smithi157 ceph-mon[32580]: pgmap v318: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:46:20.815 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-23T18:46:20.815 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-23T18:46:20.816 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-23T18:46:20.816 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-23T18:46:20.816 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-23T18:46:20.816 DEBUG:tasks.check_counter:Getting stats from b 2024-04-23T18:46:20.816 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-23T18:46:21.070 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:46:21.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:21 smithi148 ceph-mon[32311]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T18:46:21.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:21 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:21.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:21 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:21.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:21 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:21.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:21 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:21.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:21 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:21.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:21 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:21.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:21 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:46:21.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:21 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:46:21.764 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:21 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:22.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:21 smithi157 ceph-mon[32580]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T18:46:22.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:21 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:22.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:21 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:22.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:21 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:22.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:21 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:22.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:21 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:22.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:21 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:22.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:21 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:46:22.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:21 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:46:22.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:21 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:22.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:21 smithi132 ceph-mon[29178]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T18:46:22.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:21 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:22.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:21 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:22.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:21 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:22.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:21 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:22.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:21 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:22.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:21 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:46:22.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:21 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:46:22.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:21 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:46:22.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:21 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' 2024-04-23T18:46:22.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:22 smithi148 ceph-mon[32311]: pgmap v320: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:46:22.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:22 smithi148 ceph-mon[32311]: Adjusting osd_memory_target on smithi132 to 176.2M 2024-04-23T18:46:22.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:22 smithi148 ceph-mon[32311]: Unable to set osd_memory_target on smithi132 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-04-23T18:46:22.949 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-23T18:46:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:46:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 2128, 2024-04-23T18:46:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 1238, 2024-04-23T18:46:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 2575973, 2024-04-23T18:46:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 704664, 2024-04-23T18:46:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 27, 2024-04-23T18:46:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-04-23T18:46:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.255730847, 2024-04-23T18:46:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.069535618, 2024-04-23T18:46:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.114580986, 2024-04-23T18:46:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.074666980, 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1238, 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.171090467, 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000138199 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2408, 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000498696, 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000207 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 1983600, 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 220144 2024-04-23T18:46:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 6324, 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 3252, 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1910589, 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 1169858, 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 16, 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.430824537, 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.134367065, 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.289885590, 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.072028933, 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3253, 2024-04-23T18:46:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.130045275, 2024-04-23T18:46:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000039977 2024-04-23T18:46:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8920, 2024-04-23T18:46:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001236322, 2024-04-23T18:46:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000138 2024-04-23T18:46:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 318576, 2024-04-23T18:46:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 206688 2024-04-23T18:46:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:46:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 1075, 2024-04-23T18:46:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 860, 2024-04-23T18:46:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 526616, 2024-04-23T18:46:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 1013948, 2024-04-23T18:46:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 19, 2024-04-23T18:46:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 8, 2024-04-23T18:46:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.124264842, 2024-04-23T18:46:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.041281448, 2024-04-23T18:46:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.042731363, 2024-04-23T18:46:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.036082218, 2024-04-23T18:46:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 854, 2024-04-23T18:46:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.177275020, 2024-04-23T18:46:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000207581 2024-04-23T18:46:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1052, 2024-04-23T18:46:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000209342, 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000198 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 100768, 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 348352 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-23T18:46:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 395, 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.085048247, 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000215312 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 61, 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002203076, 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000036116 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 186, 2024-04-23T18:46:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 185, 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 185, 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 11.657841102, 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.063015357 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 1, 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 43, 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-23T18:46:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 29, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 7, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 22, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 19, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 1, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 3, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 3, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 2, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 238, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 216, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-04-23T18:46:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 1, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 1, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 1, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 4, 2024-04-23T18:46:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 234, 2024-04-23T18:46:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 6, 2024-04-23T18:46:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:46:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-23T18:46:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:46:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:46:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:46:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:46:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 3, 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 26, 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 45, 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 47, 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 15, 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 5, 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 45 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-23T18:46:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 10, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 1, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 13, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 12, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:46:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:46:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-23T18:46:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-23T18:46:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 2, 2024-04-23T18:46:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 24, 2024-04-23T18:46:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:46:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 1, 2024-04-23T18:46:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-23T18:46:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:46:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-23T18:46:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-23T18:46:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-23T18:46:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-23T18:46:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 175, 2024-04-23T18:46:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 101, 2024-04-23T18:46:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 101, 2024-04-23T18:46:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 74, 2024-04-23T18:46:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-23T18:46:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 45, 2024-04-23T18:46:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 8, 2024-04-23T18:46:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 7, 2024-04-23T18:46:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 7, 2024-04-23T18:46:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 1, 2024-04-23T18:46:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 1, 2024-04-23T18:46:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-23T18:46:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 1, 2024-04-23T18:46:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4445464, 2024-04-23T18:46:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 4537454, 2024-04-23T18:46:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 120, 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.221679157, 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001847326 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 24, 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 37, 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 13, 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 19, 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 24, 2024-04-23T18:46:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 5, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 29, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 47, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 18, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 3, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 32, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 29, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 56704, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 231800 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 253, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 2420, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 186, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 29, 2024-04-23T18:46:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 4842, 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 12, 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.011007744, 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000917312 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 99, 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.025989254, 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000262517 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-23T18:46:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7, 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001983871, 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000283410 2024-04-23T18:46:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000139850, 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000139850 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-23T18:46:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 10, 2024-04-23T18:46:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001899652, 2024-04-23T18:46:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000189965 2024-04-23T18:46:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-23T18:46:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 12, 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 11.477492669, 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.956457722 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 10, 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.050369531, 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.005036953 2024-04-23T18:46:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000237544, 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000237544 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5, 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002594678, 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000518935 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002846870, 2024-04-23T18:46:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001423435 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6, 2024-04-23T18:46:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.031708492, 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.005284748 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4, 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.017712556, 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004428139 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-23T18:46:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001040843, 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001040843 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-23T18:46:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8, 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.017388460, 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002173557 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7, 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.015429088, 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002204155 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-23T18:46:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 2, 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 4, 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 2, 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 2, 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 27, 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 13, 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 588, 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:46:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:46:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-23T18:46:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 311196, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 109, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 88, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 1, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 8352, 2024-04-23T18:46:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 175, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 108403, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 804, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 820, 2024-04-23T18:46:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 406196, 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 37, 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 820, 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 820, 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 20.072522083, 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.024478685 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 820, 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 966 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 820, 2024-04-23T18:46:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 4, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 816, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 23, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 31, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 4, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 124, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 46, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 539, 2024-04-23T18:46:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 62, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:46:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-23T18:46:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 99, 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 14, 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 12, 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 12, 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 123, 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 14 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-23T18:46:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 47 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 9, 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 5, 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 27, 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 12, 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-23T18:46:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 9523, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 4064574, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 9523, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 9523, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 4064574, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 820, 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 407408, 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 820, 2024-04-23T18:46:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 407408, 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 820, 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 820, 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 820, 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 820, 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 12, 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 1212, 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 12, 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 4, 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1212, 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x55645e47c500": { 2024-04-23T18:46:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 175, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 343150, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 175, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 120, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 343150, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:22.998 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-23T18:46:23.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:22 smithi157 ceph-mon[32580]: pgmap v320: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:46:23.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:22 smithi157 ceph-mon[32580]: Adjusting osd_memory_target on smithi132 to 176.2M 2024-04-23T18:46:23.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:22 smithi157 ceph-mon[32580]: Unable to set osd_memory_target on smithi132 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-04-23T18:46:23.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:22 smithi132 ceph-mon[29178]: pgmap v320: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:46:23.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:22 smithi132 ceph-mon[29178]: Adjusting osd_memory_target on smithi132 to 176.2M 2024-04-23T18:46:23.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:22 smithi132 ceph-mon[29178]: Unable to set osd_memory_target on smithi132 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-04-23T18:46:23.774 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-04-23T18:46:23.774 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-04-23T18:46:23.774 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-23T18:46:23.774 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-23T18:46:23.774 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-23T18:46:23.774 DEBUG:tasks.check_counter:Getting stats from e 2024-04-23T18:46:23.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-23T18:46:24.020 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:46:25.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:24 smithi148 ceph-mon[32311]: pgmap v321: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:46:25.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:24 smithi157 ceph-mon[32580]: pgmap v321: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:46:25.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:24 smithi132 ceph-mon[29178]: pgmap v321: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-23T18:46:25.620 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-23T18:46:25.620 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:46:25.620 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 1792, 2024-04-23T18:46:25.620 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 1771, 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 2473622, 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 621858, 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 1, 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.186666151, 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.070373758, 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.067145844, 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.045567428, 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1770, 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.068621732, 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000038769 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.621 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 1671376, 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 130544 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 10, 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 165, 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 4968, 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 441994, 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 7, 2024-04-23T18:46:25.622 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.010158796, 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.006138544, 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.002109725, 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000086862, 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 163, 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.018979212, 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000116436 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.623 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 440656 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 724, 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 726, 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 178210, 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 230239, 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.061772717, 2024-04-23T18:46:25.624 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.017657481, 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.014313183, 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.024358530, 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 724, 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.020393833, 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000028168 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:46:25.625 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3803, 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.098334750, 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000025857 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:25.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 0, 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 0, 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-23T18:46:25.627 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 0, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 31, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 13, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 18, 2024-04-23T18:46:25.628 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 13, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 2, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 0, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 0, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-04-23T18:46:25.629 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 4, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 234, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 6, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-23T18:46:25.630 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:46:25.631 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 0, 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 0 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 0, 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 0, 2024-04-23T18:46:25.632 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-23T18:46:25.633 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 0, 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 175, 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 0, 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 2, 2024-04-23T18:46:25.634 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 8, 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4445464, 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 4537454, 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 4537342, 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 175 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-23T18:46:25.635 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 17, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 30, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 13, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 12, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 17, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 5, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 31, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 37, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 6, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 0, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 0, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 51440, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 215416 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.636 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 0, 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 0, 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 0, 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.638 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.639 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-23T18:46:25.640 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.641 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.641 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.641 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-23T18:46:25.641 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.642 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.642 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.642 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-23T18:46:25.642 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.642 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.642 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.642 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-23T18:46:25.642 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.642 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-23T18:46:25.643 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.645 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.646 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-23T18:46:25.646 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.646 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.646 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.646 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-23T18:46:25.646 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.646 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.647 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 2, 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 4, 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 2, 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 3, 2024-04-23T18:46:25.649 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 0, 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 0, 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 2403, 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:46:25.650 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:46:25.651 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:46:25.651 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:46:25.651 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:46:25.651 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:46:25.651 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:46:25.651 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:46:25.651 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:46:25.651 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:46:25.651 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-23T18:46:25.651 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:46:25.651 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-23T18:46:25.652 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 189507, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 54, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:46:25.653 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 8232, 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 172, 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 81451, 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 460, 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-23T18:46:25.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 2273, 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 26, 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 4, 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 2273, 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2273, 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.877979002, 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000386264 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-23T18:46:25.655 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2273, 2024-04-23T18:46:25.656 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2274 2024-04-23T18:46:25.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.656 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 2273, 2024-04-23T18:46:25.656 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 2273, 2024-04-23T18:46:25.656 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 0, 2024-04-23T18:46:25.656 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-23T18:46:25.656 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-23T18:46:25.656 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 1461, 2024-04-23T18:46:25.656 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 0, 2024-04-23T18:46:25.656 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 809, 2024-04-23T18:46:25.656 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 0, 2024-04-23T18:46:25.656 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 0, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 0, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 2, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 0, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-23T18:46:25.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-23T18:46:25.658 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 99, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 13, 2024-04-23T18:46:25.659 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 5, 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 5, 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 0, 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 2, 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 0 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 0, 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-23T18:46:25.660 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 0, 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 0, 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 0, 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.661 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 2522, 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 2414243, 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 2522, 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2522, 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 2414243, 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:25.662 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 2273, 2024-04-23T18:46:25.663 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 343160, 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2273, 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 343160, 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 2273, 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 2273, 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2273, 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 2273, 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.665 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:46:25.666 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:25.666 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-23T18:46:25.666 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:25.666 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-23T18:46:25.666 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-23T18:46:25.666 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:25.666 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:25.666 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-23T18:46:25.666 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-23T18:46:25.667 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-04-23T18:46:25.667 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-04-23T18:46:25.667 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:25.667 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.667 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.667 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.667 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:25.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:25.667 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x561e140c26e0": { 2024-04-23T18:46:25.667 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:25.667 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:25.668 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:25.669 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-23T18:46:26.504 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-23T18:46:26.504 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-23T18:46:26.504 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-23T18:46:26.504 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:46:26.504 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:46:26.504 DEBUG:tasks.check_counter:Getting stats from h 2024-04-23T18:46:26.505 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-23T18:46:26.747 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:46:27.001 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:26 smithi148 ceph-mon[32311]: pgmap v322: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:46:27.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:26 smithi157 ceph-mon[32580]: pgmap v322: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:46:27.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:26 smithi132 ceph-mon[29178]: pgmap v322: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:46:28.500 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 295, 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 502, 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 103422, 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 229869, 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 8, 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.053690734, 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.024567972, 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.019839457, 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.004690051, 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 500, 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.043911852, 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000087823 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 713, 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000214981, 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000301 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 28, 2024-04-23T18:46:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 182, 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 8536, 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 494582, 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 15, 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.014887949, 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.006672027, 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.006053593, 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000310393, 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 180, 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.026566283, 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000147590 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 19, 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000002669, 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000140 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 5168, 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 442016 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 297, 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 287, 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1686657, 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 173164, 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 8, 2024-04-23T18:46:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.051361960, 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.015329340, 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.041388703, 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.005561635, 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 286, 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.020707997, 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000072405 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 32, 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000009085, 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000283 2024-04-23T18:46:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 1684144, 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 140336 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 46, 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001870892, 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000040671 2024-04-23T18:46:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000009674, 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000004837 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 0, 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 0, 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-23T18:46:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 11, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 10, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 10, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-04-23T18:46:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 10, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 1, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 4, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 4, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-23T18:46:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 4, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 234, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:46:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 0, 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 0 2024-04-23T18:46:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 1, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 0, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 0, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-23T18:46:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 2, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 2, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-23T18:46:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 7, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 7, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 2, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 2, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-04-23T18:46:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 4196347, 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6, 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.017848797, 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002974799 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 13, 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 13, 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 0, 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 11, 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 12, 2024-04-23T18:46:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 1, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 10, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 10, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 0, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 0, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 0, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 55344, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 223608 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 0, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 0, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 16, 2024-04-23T18:46:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 2, 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-23T18:46:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-23T18:46:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-23T18:46:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-23T18:46:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-23T18:46:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 3, 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 2, 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 0, 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 0, 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 567, 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:46:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:46:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 246343, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 80, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-23T18:46:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 8232, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 172, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 50880, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 199, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-23T18:46:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 44, 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 12525, 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 33, 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 44, 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 44, 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.252993998, 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.005749863 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-23T18:46:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 44, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 66 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 44, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 0, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 44, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 10, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 0, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 6, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 9, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-23T18:46:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 6, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 20, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-23T18:46:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-23T18:46:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 99, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 13, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 9, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 9, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 14, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-04-23T18:46:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 1 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 4 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-23T18:46:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 0, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 0, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 0, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 616, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 1725446, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 616, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-23T18:46:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 616, 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1725446, 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 44, 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 12525, 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 44, 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 12525, 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-23T18:46:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 44, 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 44, 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 44, 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 44, 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x55a78852c500": { 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 7, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 2043, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 7, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 6, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 2043, 2024-04-23T18:46:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:28.545 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:28.545 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:28.545 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:28.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:28.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:28.545 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-23T18:46:28.766 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:28 smithi148 ceph-mon[32311]: pgmap v323: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:46:29.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:28 smithi157 ceph-mon[32580]: pgmap v323: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:46:29.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:28 smithi132 ceph-mon[29178]: pgmap v323: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-23T18:46:29.345 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-23T18:46:29.345 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-23T18:46:29.345 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-23T18:46:29.345 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:46:29.345 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:46:29.345 DEBUG:tasks.check_counter:Getting stats from k 2024-04-23T18:46:29.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-23T18:46:29.591 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:46:30.748 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:30 smithi148 ceph-mon[32311]: pgmap v324: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-23T18:46:31.038 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:30 smithi157 ceph-mon[32580]: pgmap v324: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-23T18:46:31.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:30 smithi132 ceph-mon[29178]: pgmap v324: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-23T18:46:31.335 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-23T18:46:31.336 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:46:31.336 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 62, 2024-04-23T18:46:31.336 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 78, 2024-04-23T18:46:31.336 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 17187, 2024-04-23T18:46:31.336 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 97132, 2024-04-23T18:46:31.336 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 16, 2024-04-23T18:46:31.336 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2024-04-23T18:46:31.336 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.010534262, 2024-04-23T18:46:31.336 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.001936941, 2024-04-23T18:46:31.336 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.006384708, 2024-04-23T18:46:31.336 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000561305, 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 77, 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.011544120, 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000149923 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 62, 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000012158, 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000196 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 8144, 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 10016 2024-04-23T18:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.338 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:46:31.338 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 292, 2024-04-23T18:46:31.338 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 398, 2024-04-23T18:46:31.338 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1676203, 2024-04-23T18:46:31.338 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 456983, 2024-04-23T18:46:31.338 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 7, 2024-04-23T18:46:31.338 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-04-23T18:46:31.338 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.059343831, 2024-04-23T18:46:31.338 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.019415224, 2024-04-23T18:46:31.338 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.044360621, 2024-04-23T18:46:31.338 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.005653679, 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 397, 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.040658648, 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000102414 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 1672912, 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 451312 2024-04-23T18:46:31.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 307, 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 550, 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 105647, 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 478123, 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 9, 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.062796050, 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.029914888, 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.024712526, 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.003895220, 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:31.340 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 547, 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.033112306, 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000060534 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 782, 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000211605, 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000270 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 134704 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-23T18:46:31.341 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-23T18:46:31.342 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-23T18:46:31.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.342 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-23T18:46:31.342 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-23T18:46:31.342 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-23T18:46:31.342 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 55, 2024-04-23T18:46:31.342 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002014868, 2024-04-23T18:46:31.342 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000036633 2024-04-23T18:46:31.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:31.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000007037, 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000003518 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 4, 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 3, 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-23T18:46:31.343 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001948422, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000649474 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 1, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 11, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 11, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 11, 2024-04-23T18:46:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 10, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 1, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 2, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 12, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 8, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 2, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-04-23T18:46:31.345 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-23T18:46:31.346 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-04-23T18:46:31.346 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-04-23T18:46:31.346 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-04-23T18:46:31.346 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-04-23T18:46:31.346 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-23T18:46:31.346 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:46:31.346 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 1, 2024-04-23T18:46:31.346 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 4, 2024-04-23T18:46:31.346 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 234, 2024-04-23T18:46:31.346 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-04-23T18:46:31.347 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:46:31.347 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-23T18:46:31.347 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:46:31.347 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:46:31.347 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:46:31.347 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:46:31.347 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:46:31.348 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-04-23T18:46:31.348 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:46:31.348 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 2, 2024-04-23T18:46:31.348 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:46:31.348 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:46:31.348 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:46:31.349 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:46:31.349 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 2, 2024-04-23T18:46:31.349 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:46:31.349 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 0, 2024-04-23T18:46:31.349 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 0 2024-04-23T18:46:31.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.349 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-23T18:46:31.349 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 3, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 0, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 0, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-23T18:46:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:46:31.351 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:46:31.351 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-23T18:46:31.351 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-23T18:46:31.351 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 2, 2024-04-23T18:46:31.351 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 4, 2024-04-23T18:46:31.351 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:46:31.352 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-04-23T18:46:31.352 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-23T18:46:31.352 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:46:31.352 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-23T18:46:31.352 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-23T18:46:31.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.352 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-23T18:46:31.352 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-23T18:46:31.352 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 10, 2024-04-23T18:46:31.352 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-04-23T18:46:31.353 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-04-23T18:46:31.353 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 10, 2024-04-23T18:46:31.353 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-23T18:46:31.353 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-04-23T18:46:31.353 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 2, 2024-04-23T18:46:31.353 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-04-23T18:46:31.353 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-04-23T18:46:31.353 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 2, 2024-04-23T18:46:31.353 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-04-23T18:46:31.353 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-23T18:46:31.353 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-04-23T18:46:31.353 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 4197187, 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 9, 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.016075288, 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001786143 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 14, 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 14, 2024-04-23T18:46:31.354 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 0, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 13, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 14, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 1, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 11, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 12, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 1, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 2, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 2, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 52088, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 223608 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 9, 2024-04-23T18:46:31.355 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 4, 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 29, 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 5, 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001269747, 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001269747 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-23T18:46:31.356 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000678675, 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000339337 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.357 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.358 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.358 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-23T18:46:31.358 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.358 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.358 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.358 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-23T18:46:31.358 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.358 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.358 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.359 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-23T18:46:31.359 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.359 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.359 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.359 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-23T18:46:31.359 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.359 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.359 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.359 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-23T18:46:31.360 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.360 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.360 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.360 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-23T18:46:31.360 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.361 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.361 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.361 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-23T18:46:31.361 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.361 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.361 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.361 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-23T18:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-23T18:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-23T18:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.363 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.363 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-23T18:46:31.363 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.363 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.363 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.363 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-23T18:46:31.363 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.365 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.366 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.366 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-23T18:46:31.366 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.366 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.366 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.366 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-23T18:46:31.366 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.366 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.367 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.368 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.368 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-23T18:46:31.368 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.368 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.368 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.368 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:46:31.368 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:46:31.368 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-23T18:46:31.368 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.368 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 2, 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 5, 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 3, 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 2, 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 2, 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 1, 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 594, 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.369 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:46:31.370 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-23T18:46:31.371 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 247399, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 87, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-23T18:46:31.372 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 8232, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 172, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 56568, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 227, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-23T18:46:31.373 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-23T18:46:31.374 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-23T18:46:31.374 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-23T18:46:31.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.374 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-23T18:46:31.374 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-23T18:46:31.374 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-23T18:46:31.374 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 49, 2024-04-23T18:46:31.374 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 13510, 2024-04-23T18:46:31.374 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 33, 2024-04-23T18:46:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 49, 2024-04-23T18:46:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-23T18:46:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 49, 2024-04-23T18:46:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.198995264, 2024-04-23T18:46:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004061127 2024-04-23T18:46:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-23T18:46:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-23T18:46:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 49, 2024-04-23T18:46:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 73 2024-04-23T18:46:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 49, 2024-04-23T18:46:31.376 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 0, 2024-04-23T18:46:31.376 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 49, 2024-04-23T18:46:31.376 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-23T18:46:31.376 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-23T18:46:31.376 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-04-23T18:46:31.376 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 10, 2024-04-23T18:46:31.376 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 0, 2024-04-23T18:46:31.376 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 9, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 9, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 6, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 0, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 20, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:46:31.377 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-23T18:46:31.378 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:46:31.378 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-23T18:46:31.378 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-23T18:46:31.378 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-23T18:46:31.378 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:46:31.378 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-23T18:46:31.378 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:46:31.378 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-04-23T18:46:31.378 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-23T18:46:31.379 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 99, 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 13, 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 8, 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 8, 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 17, 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 2 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-23T18:46:31.380 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-04-23T18:46:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:46:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-23T18:46:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-23T18:46:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-23T18:46:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 6 2024-04-23T18:46:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-23T18:46:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-23T18:46:31.381 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:46:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-23T18:46:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 0, 2024-04-23T18:46:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 0, 2024-04-23T18:46:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 0, 2024-04-23T18:46:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-23T18:46:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:46:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-23T18:46:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 657, 2024-04-23T18:46:31.382 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 1721324, 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 657, 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 657, 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1721324, 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-23T18:46:31.383 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:31.384 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-23T18:46:31.384 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:31.384 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-23T18:46:31.384 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-23T18:46:31.384 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:31.384 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:31.384 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 49, 2024-04-23T18:46:31.384 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 13510, 2024-04-23T18:46:31.384 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 49, 2024-04-23T18:46:31.385 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 13510, 2024-04-23T18:46:31.385 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:31.385 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.385 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.385 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.385 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:31.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.385 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-23T18:46:31.385 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:31.385 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-23T18:46:31.385 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:31.385 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 49, 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 49, 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 49, 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 49, 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.386 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x55bcfea10500": { 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 10, 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 2883, 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 10, 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-23T18:46:31.388 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 9, 2024-04-23T18:46:31.389 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 2883, 2024-04-23T18:46:31.389 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-23T18:46:31.389 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-23T18:46:31.389 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-23T18:46:31.389 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-23T18:46:31.389 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:31.389 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-23T18:46:31.389 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-23T18:46:32.338 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-23T18:46:32.338 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-23T18:46:32.338 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-23T18:46:32.339 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:46:32.339 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:46:32.339 DEBUG:tasks.check_counter:Getting stats from c 2024-04-23T18:46:32.339 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-23T18:46:32.593 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:46:32.969 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:32 smithi157 ceph-mon[32580]: pgmap v325: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-04-23T18:46:33.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:32 smithi148 ceph-mon[32311]: pgmap v325: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-04-23T18:46:33.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:32 smithi132 ceph-mon[29178]: pgmap v325: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-04-23T18:46:34.462 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 999, 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 976, 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1925808, 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 369336, 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 1, 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.133966904, 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.042205886, 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.059541656, 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.032642939, 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:34.463 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 976, 2024-04-23T18:46:34.464 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.057518962, 2024-04-23T18:46:34.464 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000058933 2024-04-23T18:46:34.464 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.464 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:34.464 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.464 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.464 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.464 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.464 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1674480, 2024-04-23T18:46:34.464 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 131472 2024-04-23T18:46:34.464 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.465 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:46:34.465 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 4, 2024-04-23T18:46:34.465 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 161, 2024-04-23T18:46:34.465 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 876, 2024-04-23T18:46:34.465 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 444138, 2024-04-23T18:46:34.465 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-04-23T18:46:34.465 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-04-23T18:46:34.465 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.009784137, 2024-04-23T18:46:34.466 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.005821758, 2024-04-23T18:46:34.466 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.002199731, 2024-04-23T18:46:34.466 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000082859, 2024-04-23T18:46:34.466 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:34.466 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 158, 2024-04-23T18:46:34.466 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.014183227, 2024-04-23T18:46:34.466 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000089767 2024-04-23T18:46:34.467 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.467 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:34.467 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.467 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.467 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.467 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.467 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:46:34.468 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 443440 2024-04-23T18:46:34.468 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.468 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:46:34.468 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1426, 2024-04-23T18:46:34.468 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1428, 2024-04-23T18:46:34.468 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 365452, 2024-04-23T18:46:34.469 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 453475, 2024-04-23T18:46:34.469 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2024-04-23T18:46:34.469 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-04-23T18:46:34.469 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.096667784, 2024-04-23T18:46:34.469 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.032433824, 2024-04-23T18:46:34.469 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.023462784, 2024-04-23T18:46:34.469 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.034409307, 2024-04-23T18:46:34.470 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:34.470 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1426, 2024-04-23T18:46:34.470 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.046163790, 2024-04-23T18:46:34.470 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000032372 2024-04-23T18:46:34.470 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.470 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:34.470 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.471 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.471 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.471 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.471 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:46:34.471 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:46:34.471 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.471 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-04-23T18:46:34.471 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-04-23T18:46:34.471 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-04-23T18:46:34.472 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.472 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-04-23T18:46:34.472 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-04-23T18:46:34.472 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-04-23T18:46:34.472 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3675, 2024-04-23T18:46:34.472 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.100024292, 2024-04-23T18:46:34.473 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000027217 2024-04-23T18:46:34.473 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:34.473 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.473 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-04-23T18:46:34.473 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-04-23T18:46:34.473 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-04-23T18:46:34.473 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.474 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.474 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.474 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:34.474 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.474 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-04-23T18:46:34.474 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-04-23T18:46:34.475 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-04-23T18:46:34.475 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-04-23T18:46:34.475 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.475 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.475 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.475 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.475 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-04-23T18:46:34.475 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-04-23T18:46:34.475 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-04-23T18:46:34.475 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 1, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-04-23T18:46:34.476 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:46:34.477 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-04-23T18:46:34.478 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2024-04-23T18:46:34.478 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2024-04-23T18:46:34.478 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-04-23T18:46:34.478 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:46:34.478 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-04-23T18:46:34.478 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:46:34.478 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:46:34.478 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:46:34.478 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:46:34.478 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-04-23T18:46:34.479 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-04-23T18:46:34.480 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-04-23T18:46:34.481 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 7, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 2, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 2, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 4196347, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 4196206, 2024-04-23T18:46:34.482 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 7 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 3, 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 3, 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 0, 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 1, 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 1, 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 0, 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 0, 2024-04-23T18:46:34.483 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 0, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 0, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 50668, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 215416 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.484 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.485 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.486 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.486 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.486 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-04-23T18:46:34.486 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.486 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.486 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.486 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.486 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-04-23T18:46:34.486 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.486 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.486 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.487 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-04-23T18:46:34.488 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.489 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.489 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.489 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.489 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-04-23T18:46:34.489 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.489 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-04-23T18:46:34.490 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.491 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.492 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.493 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.494 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.495 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-04-23T18:46:34.495 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.495 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.495 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.495 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.495 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-04-23T18:46:34.495 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.495 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.495 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.495 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.495 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 3, 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 2, 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 2, 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-04-23T18:46:34.496 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 2947, 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:46:34.497 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:46:34.498 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:46:34.499 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:46:34.499 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-04-23T18:46:34.499 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:46:34.499 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:46:34.499 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:46:34.499 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-04-23T18:46:34.499 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-04-23T18:46:34.499 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 181370, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 49, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-04-23T18:46:34.500 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:46:34.501 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-04-23T18:46:34.501 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8232, 2024-04-23T18:46:34.501 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 172, 2024-04-23T18:46:34.501 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:46:34.501 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:46:34.501 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-04-23T18:46:34.501 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-04-23T18:46:34.501 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 8512, 2024-04-23T18:46:34.501 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 13, 2024-04-23T18:46:34.501 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-04-23T18:46:34.501 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-04-23T18:46:34.501 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 2176, 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 21, 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 3, 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 2176, 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2176, 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.916982114, 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000421407 2024-04-23T18:46:34.502 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2176, 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2177 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 2176, 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 2176, 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 1417, 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-04-23T18:46:34.503 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 757, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 1, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-04-23T18:46:34.504 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-04-23T18:46:34.505 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-04-23T18:46:34.506 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-04-23T18:46:34.506 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-04-23T18:46:34.506 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-04-23T18:46:34.506 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-04-23T18:46:34.506 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-04-23T18:46:34.506 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-04-23T18:46:34.506 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-04-23T18:46:34.506 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-04-23T18:46:34.506 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-04-23T18:46:34.506 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-04-23T18:46:34.506 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 99, 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 13, 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 4, 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 4, 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-04-23T18:46:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-04-23T18:46:34.508 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:46:34.508 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-04-23T18:46:34.508 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-04-23T18:46:34.508 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.508 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-04-23T18:46:34.510 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-04-23T18:46:34.511 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.511 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-04-23T18:46:34.511 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-04-23T18:46:34.511 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:46:34.511 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-04-23T18:46:34.511 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-04-23T18:46:34.511 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-04-23T18:46:34.511 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 2425, 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 2058303, 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 2425, 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2425, 2024-04-23T18:46:34.512 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2058303, 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:34.513 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2176, 2024-04-23T18:46:34.514 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 2048, 2024-04-23T18:46:34.514 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2176, 2024-04-23T18:46:34.514 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2048, 2024-04-23T18:46:34.514 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:34.514 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.514 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.514 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.514 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:34.514 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.514 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2176, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 2176, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2176, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2176, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.515 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-04-23T18:46:34.516 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-04-23T18:46:34.517 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:34.517 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.517 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.517 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.517 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:34.517 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:34.517 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55d2502ac6e0": { 2024-04-23T18:46:34.517 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:34.517 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-04-23T18:46:34.517 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:34.517 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:34.517 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:34.518 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-23T18:46:34.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:34 smithi148 ceph-mon[32311]: pgmap v326: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:34.934 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:34 smithi157 ceph-mon[32580]: pgmap v326: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:35.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:34 smithi132 ceph-mon[29178]: pgmap v326: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:35.278 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-23T18:46:35.279 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-23T18:46:35.279 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-23T18:46:35.279 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:46:35.279 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:46:35.279 DEBUG:tasks.check_counter:Getting stats from f 2024-04-23T18:46:35.279 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-23T18:46:35.523 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:46:36.961 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:36 smithi157 ceph-mon[32580]: pgmap v327: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:46:37.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:36 smithi148 ceph-mon[32311]: pgmap v327: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:46:37.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:36 smithi132 ceph-mon[29178]: pgmap v327: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:46:37.385 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-23T18:46:37.385 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:46:37.385 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 229, 2024-04-23T18:46:37.385 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 229, 2024-04-23T18:46:37.385 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 4256693, 2024-04-23T18:46:37.385 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 72062, 2024-04-23T18:46:37.385 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.024660871, 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.005765923, 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.009260491, 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.006999043, 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 228, 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.011724082, 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000051421 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 893, 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 872, 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 6034042, 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 336621, 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.119322334, 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.035204909, 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.058792042, 2024-04-23T18:46:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.029108065, 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 871, 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.040478769, 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000046473 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1672096, 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 131824 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1296, 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1454, 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 485158, 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 855108, 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 7, 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.154928654, 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.049587388, 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.040997383, 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.048631583, 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1451, 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.070120986, 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000048325 2024-04-23T18:46:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 443616 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-04-23T18:46:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3622, 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.116093133, 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000032052 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-04-23T18:46:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-04-23T18:46:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 1205, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 1205, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 1205, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 1, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:46:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:46:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-04-23T18:46:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-04-23T18:46:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 6016, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 8, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 2, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-04-23T18:46:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 12673180, 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 12670419, 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 6016 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 9, 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 1216, 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 1207, 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 7, 2024-04-23T18:46:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 16, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 9, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 1205, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 1214, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 9, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 54500, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 215416 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-04-23T18:46:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-04-23T18:46:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-04-23T18:46:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-04-23T18:46:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-04-23T18:46:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-04-23T18:46:37.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.407 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.408 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.409 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-04-23T18:46:37.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 4, 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 2, 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 4, 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 2949, 2024-04-23T18:46:37.412 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:46:37.413 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:46:37.414 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 198297, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 58, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-04-23T18:46:37.415 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8232, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 172, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 883776, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 4918, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-04-23T18:46:37.416 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 2163, 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 21, 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 3, 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 2163, 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2163, 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.235975915, 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000571417 2024-04-23T18:46:37.417 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2163, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2164 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 2163, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 2163, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 1385, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 776, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-04-23T18:46:37.418 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 1, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-04-23T18:46:37.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-04-23T18:46:37.420 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 99, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 13, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 6, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 6, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-04-23T18:46:37.421 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-04-23T18:46:37.422 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 2414, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 10542946, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 2414, 2024-04-23T18:46:37.423 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2414, 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 10542946, 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2163, 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 8479293, 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2163, 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 8479293, 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:37.425 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2163, 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 2163, 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2163, 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2163, 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.426 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-04-23T18:46:37.427 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:37.428 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.428 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.428 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.428 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:37.428 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:37.428 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55f7fc9486e0": { 2024-04-23T18:46:37.428 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:37.428 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-04-23T18:46:37.428 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:37.428 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:37.428 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:37.428 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:37.429 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:37.429 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-23T18:46:37.429 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-23T18:46:37.429 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-04-23T18:46:37.429 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-04-23T18:46:37.429 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:37.429 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:37.429 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:37.429 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:37.429 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:37.429 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:37.429 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-23T18:46:38.106 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-23T18:46:38.106 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-23T18:46:38.106 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-23T18:46:38.106 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:46:38.107 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:46:38.107 DEBUG:tasks.check_counter:Getting stats from i 2024-04-23T18:46:38.107 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-23T18:46:38.348 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:46:38.764 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:38 smithi157 ceph-mon[32580]: pgmap v328: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:39.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:38 smithi148 ceph-mon[32311]: pgmap v328: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:39.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:38 smithi132 ceph-mon[29178]: pgmap v328: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:40.084 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-23T18:46:40.084 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:46:40.084 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 7081, 2024-04-23T18:46:40.084 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 12011, 2024-04-23T18:46:40.084 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2420940, 2024-04-23T18:46:40.084 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 9602856, 2024-04-23T18:46:40.084 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 10, 2024-04-23T18:46:40.084 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.466037056, 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.200845859, 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.281818287, 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.045335819, 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 12009, 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.201581123, 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000016785 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 12264, 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.001570181, 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000128 2024-04-23T18:46:40.085 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1854720, 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 4676832 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 5377, 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 7971, 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1441290, 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 6784237, 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 10, 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 7, 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.364935762, 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.149045875, 2024-04-23T18:46:40.086 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.151652709, 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.057299200, 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 7970, 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.175665686, 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000022040 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8061, 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.001334796, 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000165 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 4080, 2024-04-23T18:46:40.087 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 129328 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1394, 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1391, 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2082437, 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 784966, 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 15, 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.114702106, 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.036416409, 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.057222355, 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.027610872, 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1389, 2024-04-23T18:46:40.088 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.041155374, 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000029629 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 55, 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000009881, 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000179 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1683424, 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 145152 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-04-23T18:46:40.089 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4821, 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.794445229, 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000372214 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 5596, 2024-04-23T18:46:40.090 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.187575089, 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000033519 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 2400, 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 2400, 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2400, 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3.260132326, 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001358388 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-04-23T18:46:40.091 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 11, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 15, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 15, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 13, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 5, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-04-23T18:46:40.092 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 3, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 4810, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 2406, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 3, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 1, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 1, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 1, 2024-04-23T18:46:40.093 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 4, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 234, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-04-23T18:46:40.094 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 6, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 3595, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 2403, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 1199, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 8, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 2397 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-04-23T18:46:40.095 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 4, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 1199, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 1199, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-04-23T18:46:40.096 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 2, 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 4, 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 1, 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 6016, 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-04-23T18:46:40.097 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 6016, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 8, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 8, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 2, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 12673180, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-04-23T18:46:40.098 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2381, 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3.422477461, 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001437411 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 19, 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 1220, 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 1201, 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 18, 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 21, 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 3, 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 15, 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 1220, 2024-04-23T18:46:40.099 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 1205, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 1202, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 1202, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 56620, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 223608 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 4806, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 1, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 2400, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 24, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 2405, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1198, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.226395029, 2024-04-23T18:46:40.100 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001023702 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000921797, 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000921797 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-04-23T18:46:40.101 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2, 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000743489, 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000371744 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.102 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.103 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.104 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.104 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-04-23T18:46:40.104 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.104 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.104 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.104 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.104 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-04-23T18:46:40.104 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.104 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.105 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.105 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.105 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-04-23T18:46:40.105 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.105 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.105 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.105 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.105 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-04-23T18:46:40.105 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.105 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.105 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-04-23T18:46:40.106 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-04-23T18:46:40.107 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.108 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.109 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.110 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1199, 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.032072011, 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001694805 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.111 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 4, 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 2, 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 2, 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 2212, 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 1106, 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 594, 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.112 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:46:40.113 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:46:40.114 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 291127, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 120, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 528, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 6, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:46:40.115 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8232, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 172, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 86317, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 370, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.116 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 5965, 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 8711556, 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 33, 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 5965, 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 5965, 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 8.924826412, 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001496198 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-04-23T18:46:40.117 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 5965, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 6013 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 5965, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 21, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 5944, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 0, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 10, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 21, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 2402, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 17, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-04-23T18:46:40.118 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 3484, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 0, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 20, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-04-23T18:46:40.119 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-04-23T18:46:40.120 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 99, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 13, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 12, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 12, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 49, 2024-04-23T18:46:40.121 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 0, 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 10 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 6 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-04-23T18:46:40.122 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 8, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 3, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 3470, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 1199, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 13848, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 4451807, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:40.123 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 13848, 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 13848, 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 4451807, 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-23T18:46:40.124 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 5965, 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 8832655, 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 5965, 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 8832655, 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:40.125 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 5965, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 5965, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 5965, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 5965, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.126 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 1199, 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 121099, 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 1199, 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-23T18:46:40.127 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 21, 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 121099, 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x558695e1a500": { 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 6016, 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 8478876, 2024-04-23T18:46:40.128 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:40.129 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 6016, 2024-04-23T18:46:40.129 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-23T18:46:40.129 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-23T18:46:40.129 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2381, 2024-04-23T18:46:40.129 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 8478876, 2024-04-23T18:46:40.129 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:40.129 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:40.129 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:40.129 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:40.129 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:40.129 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:40.129 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-23T18:46:40.788 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:40 smithi157 ceph-mon[32580]: pgmap v329: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:46:40.912 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-04-23T18:46:40.912 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-04-23T18:46:40.912 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-04-23T18:46:40.912 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:46:40.912 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:46:40.912 DEBUG:tasks.check_counter:Getting stats from l 2024-04-23T18:46:40.912 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c34642b4-019e-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-23T18:46:41.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:40 smithi148 ceph-mon[32311]: pgmap v329: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:46:41.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:40 smithi132 ceph-mon[29178]: pgmap v329: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:46:41.153 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:46:42.887 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:42 smithi157 ceph-mon[32580]: pgmap v330: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:42.915 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-23T18:46:42.915 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:46:42.915 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 406, 2024-04-23T18:46:42.916 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 542, 2024-04-23T18:46:42.916 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 133257, 2024-04-23T18:46:42.916 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 348002, 2024-04-23T18:46:42.916 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 9, 2024-04-23T18:46:42.916 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-04-23T18:46:42.916 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.052810603, 2024-04-23T18:46:42.916 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.020614836, 2024-04-23T18:46:42.917 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.022801822, 2024-04-23T18:46:42.917 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.005495597, 2024-04-23T18:46:42.917 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:42.917 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 541, 2024-04-23T18:46:42.917 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.036165881, 2024-04-23T18:46:42.917 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000066850 2024-04-23T18:46:42.918 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.918 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:42.918 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 866, 2024-04-23T18:46:42.918 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000293958, 2024-04-23T18:46:42.918 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000339 2024-04-23T18:46:42.918 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.919 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3920, 2024-04-23T18:46:42.919 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 1424 2024-04-23T18:46:42.919 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.919 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:46:42.919 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 328, 2024-04-23T18:46:42.919 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 371, 2024-04-23T18:46:42.919 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1675527, 2024-04-23T18:46:42.919 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 246309, 2024-04-23T18:46:42.919 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 13, 2024-04-23T18:46:42.919 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2024-04-23T18:46:42.920 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.064093099, 2024-04-23T18:46:42.920 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.019328519, 2024-04-23T18:46:42.920 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.049490967, 2024-04-23T18:46:42.920 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.005867477, 2024-04-23T18:46:42.920 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:42.920 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 370, 2024-04-23T18:46:42.920 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.025544943, 2024-04-23T18:46:42.921 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000069040 2024-04-23T18:46:42.921 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.921 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:42.921 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 109, 2024-04-23T18:46:42.921 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000029153, 2024-04-23T18:46:42.921 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000267 2024-04-23T18:46:42.921 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.921 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1672160, 2024-04-23T18:46:42.921 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 165216 2024-04-23T18:46:42.921 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 23, 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 186, 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 5913, 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 455402, 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 9, 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.012634736, 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.006836638, 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.003508601, 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000399623, 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 183, 2024-04-23T18:46:42.922 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.024814553, 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000135598 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8, 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000001415, 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000176 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 446912 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.923 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 55, 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.001903054, 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000034600 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2, 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000012226, 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000006113 2024-04-23T18:46:42.924 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:42.925 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.925 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-04-23T18:46:42.925 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 9, 2024-04-23T18:46:42.925 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 5, 2024-04-23T18:46:42.925 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-04-23T18:46:42.925 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 5, 2024-04-23T18:46:42.925 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.001476083, 2024-04-23T18:46:42.925 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000295216 2024-04-23T18:46:42.925 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.925 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-04-23T18:46:42.925 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 4, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 11, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 14, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 14, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 14, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 2, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 3, 2024-04-23T18:46:42.926 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 2, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 14, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 11, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 1, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-04-23T18:46:42.927 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:46:42.928 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-04-23T18:46:42.928 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 4, 2024-04-23T18:46:42.928 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 234, 2024-04-23T18:46:42.928 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-04-23T18:46:42.928 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:46:42.928 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-04-23T18:46:42.928 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:46:42.928 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:46:42.928 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:46:42.928 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:46:42.929 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:46:42.929 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-04-23T18:46:42.929 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:46:42.929 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 5, 2024-04-23T18:46:42.929 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 5, 2024-04-23T18:46:42.929 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:46:42.929 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:46:42.929 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:46:42.929 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 5, 2024-04-23T18:46:42.929 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 1 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 2, 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-04-23T18:46:42.930 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 2, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 10, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-04-23T18:46:42.931 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 8, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 8, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 2, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 2, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 2, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-04-23T18:46:42.932 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 4196655, 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 7, 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.011319441, 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001617063 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 17, 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 17, 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 0, 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 15, 2024-04-23T18:46:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 16, 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 1, 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 14, 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 14, 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 0, 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 3, 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 3, 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 52712, 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 223608 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 10, 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-04-23T18:46:42.934 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 9, 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 25, 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 10, 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4, 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.001330146, 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000332536 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.935 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.936 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000145937, 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000145937 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.937 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.938 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.939 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-04-23T18:46:42.939 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.939 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-04-23T18:46:42.940 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-04-23T18:46:42.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.942 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.943 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.944 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.945 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 4, 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 2, 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 2, 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-04-23T18:46:42.946 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 4, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 1, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 612, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:46:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:46:42.948 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 271991, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 97, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 176, 2024-04-23T18:46:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 2, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8232, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 172, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 68797, 2024-04-23T18:46:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 295, 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 46, 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 13148, 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 33, 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 46, 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 46, 2024-04-23T18:46:42.951 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.233995475, 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.005086858 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 46, 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 70 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 46, 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 0, 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 46, 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 0, 2024-04-23T18:46:42.952 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 10, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 0, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 7, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 8, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 6, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 0, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 20, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-04-23T18:46:42.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:46:42.954 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-04-23T18:46:42.954 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-04-23T18:46:42.954 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-04-23T18:46:42.954 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:46:42.954 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-04-23T18:46:42.954 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:46:42.954 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-04-23T18:46:42.954 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-04-23T18:46:42.954 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-04-23T18:46:42.954 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-04-23T18:46:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 99, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 13, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 9, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 9, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 18, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 0, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 1 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-04-23T18:46:42.956 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 12 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:46:42.957 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 753, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1727996, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 753, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 753, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1727996, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.958 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 46, 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 13148, 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 46, 2024-04-23T18:46:42.959 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 13148, 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:42.960 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 46, 2024-04-23T18:46:42.961 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 46, 2024-04-23T18:46:42.961 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 46, 2024-04-23T18:46:42.961 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 46, 2024-04-23T18:46:42.961 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:42.961 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.961 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.961 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.961 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:42.961 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.961 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:46:42.961 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:42.961 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:42.962 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55e0c8484500": { 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 8, 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 2351, 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 8, 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 7, 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2351, 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-04-23T18:46:42.963 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-04-23T18:46:42.964 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-04-23T18:46:42.964 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-04-23T18:46:42.964 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:42.964 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-23T18:46:42.964 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-23T18:46:43.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:42 smithi148 ceph-mon[32311]: pgmap v330: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:43.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:42 smithi132 ceph-mon[29178]: pgmap v330: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:43.774 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-23T18:46:43.775 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-23T18:46:43.775 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-23T18:46:43.775 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:46:43.775 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:46:43.776 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-23T18:46:43.794 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-23T18:46:43.794 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:46:43.794 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T18:46:43.817 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-23T18:46:43.817 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T18:46:43.879 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:46:44.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.041+0000 7ff0b9aae640 1 -- 172.21.15.132:0/551176847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b4107300 msgr2=0x7ff0b410eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.041+0000 7ff0b9aae640 1 --2- 172.21.15.132:0/551176847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b4107300 0x7ff0b410eeb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009ca0 tx=0x7ff0ac02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.041+0000 7ff0b9aae640 1 -- 172.21.15.132:0/551176847 shutdown_connections 2024-04-23T18:46:44.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.041+0000 7ff0b9aae640 1 --2- 172.21.15.132:0/551176847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b406b2b0 0x7ff0b4106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.041+0000 7ff0b9aae640 1 --2- 172.21.15.132:0/551176847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b4107300 0x7ff0b410eeb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.041+0000 7ff0b9aae640 1 --2- 172.21.15.132:0/551176847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b406a900 0x7ff0b406ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.041+0000 7ff0b9aae640 1 -- 172.21.15.132:0/551176847 >> 172.21.15.132:0/551176847 conn(0x7ff0b4077f20 msgr2=0x7ff0b4078330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:44.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.041+0000 7ff0b9aae640 1 -- 172.21.15.132:0/551176847 shutdown_connections 2024-04-23T18:46:44.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.041+0000 7ff0b9aae640 1 -- 172.21.15.132:0/551176847 wait complete. 2024-04-23T18:46:44.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.041+0000 7ff0b9aae640 1 Processor -- start 2024-04-23T18:46:44.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.041+0000 7ff0b9aae640 1 -- start start 2024-04-23T18:46:44.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b9aae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b406a900 0x7ff0b41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b9aae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4107300 0x7ff0b4198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b9aae640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b419cf80 0x7ff0b419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b9aae640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b4112200 con 0x7ff0b419cf80 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b9aae640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b4112080 con 0x7ff0b4107300 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b9aae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b4112380 con 0x7ff0b406a900 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b3fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b419cf80 0x7ff0b419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b3fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b419cf80 0x7ff0b419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37760/0 (socket says 172.21.15.132:37760) 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b3fff640 1 -- 172.21.15.132:0/1615377980 learned_addr learned my addr 172.21.15.132:0/1615377980 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b37fe640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b406a900 0x7ff0b41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b2ffd640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4107300 0x7ff0b4198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b3fff640 1 -- 172.21.15.132:0/1615377980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b406a900 msgr2=0x7ff0b41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b3fff640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b406a900 0x7ff0b41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b3fff640 1 -- 172.21.15.132:0/1615377980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4107300 msgr2=0x7ff0b4198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b3fff640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4107300 0x7ff0b4198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b3fff640 1 -- 172.21.15.132:0/1615377980 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b419da70 con 0x7ff0b419cf80 2024-04-23T18:46:44.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b2ffd640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4107300 0x7ff0b4198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:44.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b3fff640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b419cf80 0x7ff0b419d430 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0098d0 tx=0x7ff0ac0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:44.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b0ff9640 1 -- 172.21.15.132:0/1615377980 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac003b70 con 0x7ff0b419cf80 2024-04-23T18:46:44.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b9aae640 1 -- 172.21.15.132:0/1615377980 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b419dd00 con 0x7ff0b419cf80 2024-04-23T18:46:44.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b0ff9640 1 -- 172.21.15.132:0/1615377980 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0ac003d10 con 0x7ff0b419cf80 2024-04-23T18:46:44.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b0ff9640 1 -- 172.21.15.132:0/1615377980 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac02fa20 con 0x7ff0b419cf80 2024-04-23T18:46:44.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.042+0000 7ff0b9aae640 1 -- 172.21.15.132:0/1615377980 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b41a5500 con 0x7ff0b419cf80 2024-04-23T18:46:44.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.043+0000 7ff0b9aae640 1 -- 172.21.15.132:0/1615377980 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b40691c0 con 0x7ff0b419cf80 2024-04-23T18:46:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.045+0000 7ff0b0ff9640 1 -- 172.21.15.132:0/1615377980 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7ff0ac02fbe0 con 0x7ff0b419cf80 2024-04-23T18:46:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.045+0000 7ff0b0ff9640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff09c0a3d20 0x7ff09c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.045+0000 7ff0b37fe640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff09c0a3d20 0x7ff09c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.045+0000 7ff0b0ff9640 1 -- 172.21.15.132:0/1615377980 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7ff0ac0f1db0 con 0x7ff0b419cf80 2024-04-23T18:46:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.046+0000 7ff0b37fe640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff09c0a3d20 0x7ff09c0a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400bbe0 tx=0x7ff0a400b390 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:44.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.049+0000 7ff0b0ff9640 1 -- 172.21.15.132:0/1615377980 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff0ac0be520 con 0x7ff0b419cf80 2024-04-23T18:46:44.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.226+0000 7ff0b9aae640 1 -- 172.21.15.132:0/1615377980 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0b406b2b0 con 0x7ff0b419cf80 2024-04-23T18:46:44.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.227+0000 7ff0b0ff9640 1 -- 172.21.15.132:0/1615377980 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23318 (secure 0 0 0) 0x7ff0ac0bde20 con 0x7ff0b419cf80 2024-04-23T18:46:44.219 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:46:44.219 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":99,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:46:20.673047+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/2124409853":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/3095429281":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/899407689":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1857586348":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:46:44.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.229+0000 7ff0b9aae640 1 -- 172.21.15.132:0/1615377980 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff09c0a3d20 msgr2=0x7ff09c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.229+0000 7ff0b9aae640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff09c0a3d20 0x7ff09c0a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400bbe0 tx=0x7ff0a400b390 comp rx=0 tx=0).stop 2024-04-23T18:46:44.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.229+0000 7ff0b9aae640 1 -- 172.21.15.132:0/1615377980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b419cf80 msgr2=0x7ff0b419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.229+0000 7ff0b9aae640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b419cf80 0x7ff0b419d430 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0098d0 tx=0x7ff0ac0317b0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.230+0000 7ff0b9aae640 1 -- 172.21.15.132:0/1615377980 shutdown_connections 2024-04-23T18:46:44.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.230+0000 7ff0b9aae640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff09c0a3d20 0x7ff09c0a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.230+0000 7ff0b9aae640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b419cf80 0x7ff0b419d430 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.230+0000 7ff0b9aae640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4107300 0x7ff0b4198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.230+0000 7ff0b9aae640 1 --2- 172.21.15.132:0/1615377980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b406a900 0x7ff0b41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.230+0000 7ff0b9aae640 1 -- 172.21.15.132:0/1615377980 >> 172.21.15.132:0/1615377980 conn(0x7ff0b4077f20 msgr2=0x7ff0b40fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:44.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.230+0000 7ff0b9aae640 1 -- 172.21.15.132:0/1615377980 shutdown_connections 2024-04-23T18:46:44.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.230+0000 7ff0b9aae640 1 -- 172.21.15.132:0/1615377980 wait complete. 2024-04-23T18:46:44.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:29:20.132451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:33:58.407521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:33:59.518206+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:46:44.233 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:46:44.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.343+0000 7f2bec7fe640 1 -- 172.21.15.132:0/1182379068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4103af0 msgr2=0x7f2be410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.343+0000 7f2bec7fe640 1 --2- 172.21.15.132:0/1182379068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4103af0 0x7f2be410bfd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009900 tx=0x7f2be002a510 comp rx=0 tx=0).stop 2024-04-23T18:46:44.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 -- 172.21.15.132:0/1182379068 shutdown_connections 2024-04-23T18:46:44.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 --2- 172.21.15.132:0/1182379068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be410c510 0x7f2be410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 --2- 172.21.15.132:0/1182379068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4103af0 0x7f2be410bfd0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 --2- 172.21.15.132:0/1182379068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2be41031d0 0x7f2be41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 -- 172.21.15.132:0/1182379068 >> 172.21.15.132:0/1182379068 conn(0x7f2be4077f50 msgr2=0x7f2be4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 -- 172.21.15.132:0/1182379068 shutdown_connections 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 -- 172.21.15.132:0/1182379068 wait complete. 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 Processor -- start 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 -- start start 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2be41031d0 0x7f2be41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4103af0 0x7f2be41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be410c510 0x7f2be419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2be41121a0 con 0x7f2be4103af0 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4112020 con 0x7f2be41031d0 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2bec7fe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4112320 con 0x7f2be410c510 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.344+0000 7f2be9d99640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4103af0 0x7f2be41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2be9d99640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4103af0 0x7f2be41a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37780/0 (socket says 172.21.15.132:37780) 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2be9d99640 1 -- 172.21.15.132:0/3934952834 learned_addr learned my addr 172.21.15.132:0/3934952834 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:44.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2bead9b640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be410c510 0x7f2be419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2bea59a640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2be41031d0 0x7f2be41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2be9d99640 1 -- 172.21.15.132:0/3934952834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be410c510 msgr2=0x7f2be419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2be9d99640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be410c510 0x7f2be419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2be9d99640 1 -- 172.21.15.132:0/3934952834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2be41031d0 msgr2=0x7f2be41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2be9d99640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2be41031d0 0x7f2be41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2be9d99640 1 -- 172.21.15.132:0/3934952834 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be419af50 con 0x7f2be4103af0 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2be9d99640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4103af0 0x7f2be41a0b50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2be0005940 tx=0x7f2be0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2bdb7fe640 1 -- 172.21.15.132:0/3934952834 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be003f030 con 0x7f2be4103af0 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2bdb7fe640 1 -- 172.21.15.132:0/3934952834 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be0003aa0 con 0x7f2be4103af0 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2bec7fe640 1 -- 172.21.15.132:0/3934952834 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be419b1e0 con 0x7f2be4103af0 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2bdb7fe640 1 -- 172.21.15.132:0/3934952834 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0035070 con 0x7f2be4103af0 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.345+0000 7f2bec7fe640 1 -- 172.21.15.132:0/3934952834 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be41a7460 con 0x7f2be4103af0 2024-04-23T18:46:44.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.346+0000 7f2bec7fe640 1 -- 172.21.15.132:0/3934952834 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be40698e0 con 0x7f2be4103af0 2024-04-23T18:46:44.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.347+0000 7f2bdb7fe640 1 -- 172.21.15.132:0/3934952834 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f2be0042030 con 0x7f2be4103af0 2024-04-23T18:46:44.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.348+0000 7f2bdb7fe640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f2bd00a3d20 0x7f2bd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.348+0000 7f2bea59a640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f2bd00a3d20 0x7f2bd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.348+0000 7f2bdb7fe640 1 -- 172.21.15.132:0/3934952834 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f2be00f10c0 con 0x7f2be4103af0 2024-04-23T18:46:44.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.348+0000 7f2bea59a640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f2bd00a3d20 0x7f2bd00a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40058c0 tx=0x7f2bd400a2d0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:44.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.351+0000 7f2bdb7fe640 1 -- 172.21.15.132:0/3934952834 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2be00bd0d0 con 0x7f2be4103af0 2024-04-23T18:46:44.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.527+0000 7f2bec7fe640 1 -- 172.21.15.132:0/3934952834 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2be41a7630 con 0x7f2be4103af0 2024-04-23T18:46:44.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.527+0000 7f2bdb7fe640 1 -- 172.21.15.132:0/3934952834 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f2be00c13d0 con 0x7f2be4103af0 2024-04-23T18:46:44.518 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-04-23T18:46:44.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.529+0000 7f2bec7fe640 1 -- 172.21.15.132:0/3934952834 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f2bd00a3d20 msgr2=0x7f2bd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.529+0000 7f2bec7fe640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f2bd00a3d20 0x7f2bd00a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40058c0 tx=0x7f2bd400a2d0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.529+0000 7f2bec7fe640 1 -- 172.21.15.132:0/3934952834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4103af0 msgr2=0x7f2be41a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.529+0000 7f2bec7fe640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4103af0 0x7f2be41a0b50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2be0005940 tx=0x7f2be0004270 comp rx=0 tx=0).stop 2024-04-23T18:46:44.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.529+0000 7f2bec7fe640 1 -- 172.21.15.132:0/3934952834 shutdown_connections 2024-04-23T18:46:44.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.529+0000 7f2bec7fe640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f2bd00a3d20 0x7f2bd00a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.529+0000 7f2bec7fe640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be410c510 0x7f2be419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.529+0000 7f2bec7fe640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4103af0 0x7f2be41a0b50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.529+0000 7f2bec7fe640 1 --2- 172.21.15.132:0/3934952834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2be41031d0 0x7f2be41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.529+0000 7f2bec7fe640 1 -- 172.21.15.132:0/3934952834 >> 172.21.15.132:0/3934952834 conn(0x7f2be4077f50 msgr2=0x7f2be4108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:44.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.529+0000 7f2bec7fe640 1 -- 172.21.15.132:0/3934952834 shutdown_connections 2024-04-23T18:46:44.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.529+0000 7f2bec7fe640 1 -- 172.21.15.132:0/3934952834 wait complete. 2024-04-23T18:46:44.532 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:46:44.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.643+0000 7fbdc954a640 1 -- 172.21.15.132:0/2712527362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc41021a0 msgr2=0x7fbdc4102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.643+0000 7fbdc954a640 1 --2- 172.21.15.132:0/2712527362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc41021a0 0x7fbdc4102580 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009900 tx=0x7fbdb402a410 comp rx=0 tx=0).stop 2024-04-23T18:46:44.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.643+0000 7fbdc954a640 1 -- 172.21.15.132:0/2712527362 shutdown_connections 2024-04-23T18:46:44.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.643+0000 7fbdc954a640 1 --2- 172.21.15.132:0/2712527362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc4102b50 0x7fbdc4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.643+0000 7fbdc954a640 1 --2- 172.21.15.132:0/2712527362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdc4107270 0x7fbdc410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.643+0000 7fbdc954a640 1 --2- 172.21.15.132:0/2712527362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc41021a0 0x7fbdc4102580 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.643+0000 7fbdc954a640 1 -- 172.21.15.132:0/2712527362 >> 172.21.15.132:0/2712527362 conn(0x7fbdc40793e0 msgr2=0x7fbdc40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:44.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.643+0000 7fbdc954a640 1 -- 172.21.15.132:0/2712527362 shutdown_connections 2024-04-23T18:46:44.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.643+0000 7fbdc954a640 1 -- 172.21.15.132:0/2712527362 wait complete. 2024-04-23T18:46:44.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.643+0000 7fbdc954a640 1 Processor -- start 2024-04-23T18:46:44.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc954a640 1 -- start start 2024-04-23T18:46:44.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc954a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc41021a0 0x7fbdc4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc954a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc4107270 0x7fbdc4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc954a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdc419cff0 0x7fbdc419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc954a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41121b0 con 0x7fbdc4107270 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc954a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc4112030 con 0x7fbdc419cff0 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc954a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc4112330 con 0x7fbdc41021a0 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc27fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc4107270 0x7fbdc4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc27fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc4107270 0x7fbdc4198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37804/0 (socket says 172.21.15.132:37804) 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc27fc640 1 -- 172.21.15.132:0/2656798858 learned_addr learned my addr 172.21.15.132:0/2656798858 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc27fc640 1 -- 172.21.15.132:0/2656798858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc41021a0 msgr2=0x7fbdc4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc2ffd640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc41021a0 0x7fbdc4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc27fc640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc41021a0 0x7fbdc4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc27fc640 1 -- 172.21.15.132:0/2656798858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdc419cff0 msgr2=0x7fbdc419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc27fc640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdc419cff0 0x7fbdc419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc27fc640 1 -- 172.21.15.132:0/2656798858 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc419dae0 con 0x7fbdc4107270 2024-04-23T18:46:44.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc27fc640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc4107270 0x7fbdc4198c70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8005fd0 tx=0x7fbdb800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:44.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdabfff640 1 -- 172.21.15.132:0/2656798858 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8017070 con 0x7fbdc4107270 2024-04-23T18:46:44.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc954a640 1 -- 172.21.15.132:0/2656798858 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc419ddd0 con 0x7fbdc4107270 2024-04-23T18:46:44.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdabfff640 1 -- 172.21.15.132:0/2656798858 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdb8009c30 con 0x7fbdc4107270 2024-04-23T18:46:44.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdabfff640 1 -- 172.21.15.132:0/2656798858 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8003d30 con 0x7fbdc4107270 2024-04-23T18:46:44.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.644+0000 7fbdc954a640 1 -- 172.21.15.132:0/2656798858 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc41a55c0 con 0x7fbdc4107270 2024-04-23T18:46:44.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.645+0000 7fbdc954a640 1 -- 172.21.15.132:0/2656798858 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc40691c0 con 0x7fbdc4107270 2024-04-23T18:46:44.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.646+0000 7fbdabfff640 1 -- 172.21.15.132:0/2656798858 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fbdb8003ed0 con 0x7fbdc4107270 2024-04-23T18:46:44.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.647+0000 7fbdabfff640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fbdac0a3d20 0x7fbdac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.647+0000 7fbdc2ffd640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fbdac0a3d20 0x7fbdac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.647+0000 7fbdabfff640 1 -- 172.21.15.132:0/2656798858 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7fbdb80d0d20 con 0x7fbdc4107270 2024-04-23T18:46:44.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.648+0000 7fbdc2ffd640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fbdac0a3d20 0x7fbdac0a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4006010 tx=0x7fbdb4000bc0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:44.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.651+0000 7fbdabfff640 1 -- 172.21.15.132:0/2656798858 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbdb8020030 con 0x7fbdc4107270 2024-04-23T18:46:44.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.831+0000 7fbdc954a640 1 -- 172.21.15.132:0/2656798858 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbdc4102b50 con 0x7fbdc4107270 2024-04-23T18:46:44.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.831+0000 7fbdabfff640 1 -- 172.21.15.132:0/2656798858 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7fbdb8009de0 con 0x7fbdc4107270 2024-04-23T18:46:44.822 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-04-23T18:46:44.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.833+0000 7fbdc954a640 1 -- 172.21.15.132:0/2656798858 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fbdac0a3d20 msgr2=0x7fbdac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.833+0000 7fbdc954a640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fbdac0a3d20 0x7fbdac0a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4006010 tx=0x7fbdb4000bc0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.833+0000 7fbdc954a640 1 -- 172.21.15.132:0/2656798858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc4107270 msgr2=0x7fbdc4198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.833+0000 7fbdc954a640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc4107270 0x7fbdc4198c70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8005fd0 tx=0x7fbdb800dde0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.833+0000 7fbdc954a640 1 -- 172.21.15.132:0/2656798858 shutdown_connections 2024-04-23T18:46:44.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.833+0000 7fbdc954a640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fbdac0a3d20 0x7fbdac0a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.833+0000 7fbdc954a640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdc419cff0 0x7fbdc419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.833+0000 7fbdc954a640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc4107270 0x7fbdc4198c70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.833+0000 7fbdc954a640 1 --2- 172.21.15.132:0/2656798858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc41021a0 0x7fbdc4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.833+0000 7fbdc954a640 1 -- 172.21.15.132:0/2656798858 >> 172.21.15.132:0/2656798858 conn(0x7fbdc40793e0 msgr2=0x7fbdc4100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:44.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.833+0000 7fbdc954a640 1 -- 172.21.15.132:0/2656798858 shutdown_connections 2024-04-23T18:46:44.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.834+0000 7fbdc954a640 1 -- 172.21.15.132:0/2656798858 wait complete. 2024-04-23T18:46:44.834 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:44 smithi132 ceph-mon[29178]: pgmap v331: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:44.835 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:44 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1615377980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:46:44.835 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:44 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3934952834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:46:44.836 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:46:44.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.946+0000 7f1feb568640 1 -- 172.21.15.132:0/3627910760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fe41010a0 msgr2=0x7f1fe4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.946+0000 7f1feb568640 1 --2- 172.21.15.132:0/3627910760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fe41010a0 0x7f1fe4101480 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009900 tx=0x7f1fe002a410 comp rx=0 tx=0).stop 2024-04-23T18:46:44.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.946+0000 7f1feb568640 1 -- 172.21.15.132:0/3627910760 shutdown_connections 2024-04-23T18:46:44.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.946+0000 7f1feb568640 1 --2- 172.21.15.132:0/3627910760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe4101a50 0x7f1fe4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 --2- 172.21.15.132:0/3627910760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fe4106170 0x7f1fe410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 --2- 172.21.15.132:0/3627910760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fe41010a0 0x7f1fe4101480 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 -- 172.21.15.132:0/3627910760 >> 172.21.15.132:0/3627910760 conn(0x7f1fe4077eb0 msgr2=0x7f1fe40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:44.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 -- 172.21.15.132:0/3627910760 shutdown_connections 2024-04-23T18:46:44.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 -- 172.21.15.132:0/3627910760 wait complete. 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 Processor -- start 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 -- start start 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe41010a0 0x7f1fe4199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fe4106170 0x7f1fe4199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fe419a100 0x7f1fe41a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe41110d0 con 0x7f1fe419a100 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4110f50 con 0x7f1fe4106170 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.947+0000 7f1feb568640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4111250 con 0x7f1fe41010a0 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fe9b05640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fe419a100 0x7f1fe41a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fe9304640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe41010a0 0x7f1fe4199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fe9b05640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fe419a100 0x7f1fe41a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37832/0 (socket says 172.21.15.132:37832) 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fe9b05640 1 -- 172.21.15.132:0/3087677539 learned_addr learned my addr 172.21.15.132:0/3087677539 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fe8b03640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fe4106170 0x7f1fe4199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fe9b05640 1 -- 172.21.15.132:0/3087677539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe41010a0 msgr2=0x7f1fe4199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fe9b05640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe41010a0 0x7f1fe4199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fe9b05640 1 -- 172.21.15.132:0/3087677539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fe4106170 msgr2=0x7f1fe4199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:44.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fe9b05640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fe4106170 0x7f1fe4199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fe9b05640 1 -- 172.21.15.132:0/3087677539 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe41a4eb0 con 0x7f1fe419a100 2024-04-23T18:46:44.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fe9b05640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fe419a100 0x7f1fe41a46b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00cad0 tx=0x7f1fdc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:44.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fda7fc640 1 -- 172.21.15.132:0/3087677539 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00ae20 con 0x7f1fe419a100 2024-04-23T18:46:44.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1feb568640 1 -- 172.21.15.132:0/3087677539 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe41a51a0 con 0x7f1fe419a100 2024-04-23T18:46:44.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fda7fc640 1 -- 172.21.15.132:0/3087677539 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fdc005020 con 0x7f1fe419a100 2024-04-23T18:46:44.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1fda7fc640 1 -- 172.21.15.132:0/3087677539 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00f6b0 con 0x7f1fe419a100 2024-04-23T18:46:44.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.948+0000 7f1feb568640 1 -- 172.21.15.132:0/3087677539 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe41a56f0 con 0x7f1fe419a100 2024-04-23T18:46:44.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.949+0000 7f1feb568640 1 -- 172.21.15.132:0/3087677539 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe40691c0 con 0x7f1fe419a100 2024-04-23T18:46:44.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.950+0000 7f1fda7fc640 1 -- 172.21.15.132:0/3087677539 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f1fdc0051d0 con 0x7f1fe419a100 2024-04-23T18:46:44.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.951+0000 7f1fda7fc640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1fd00a3d20 0x7f1fd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.951+0000 7f1fe9304640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1fd00a3d20 0x7f1fd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.951+0000 7f1fda7fc640 1 -- 172.21.15.132:0/3087677539 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f1fdc0d08a0 con 0x7f1fe419a100 2024-04-23T18:46:44.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.951+0000 7f1fe9304640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1fd00a3d20 0x7f1fd00a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0031440 tx=0x7f1fe0005b50 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:44.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:44.955+0000 7f1fda7fc640 1 -- 172.21.15.132:0/3087677539 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fdc014030 con 0x7f1fe419a100 2024-04-23T18:46:45.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:44 smithi148 ceph-mon[32311]: pgmap v331: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:45.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:44 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1615377980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:46:45.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:44 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3934952834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:46:45.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:44 smithi157 ceph-mon[32580]: pgmap v331: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:45.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:44 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1615377980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:46:45.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:44 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3934952834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:46:45.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.131+0000 7f1feb568640 1 -- 172.21.15.132:0/3087677539 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1fe41093a0 con 0x7f1fe419a100 2024-04-23T18:46:45.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.131+0000 7f1fda7fc640 1 -- 172.21.15.132:0/3087677539 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f1fdc09cf00 con 0x7f1fe419a100 2024-04-23T18:46:45.122 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-04-23T18:46:45.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.133+0000 7f1feb568640 1 -- 172.21.15.132:0/3087677539 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1fd00a3d20 msgr2=0x7f1fd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:45.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.133+0000 7f1feb568640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1fd00a3d20 0x7f1fd00a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0031440 tx=0x7f1fe0005b50 comp rx=0 tx=0).stop 2024-04-23T18:46:45.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.133+0000 7f1feb568640 1 -- 172.21.15.132:0/3087677539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fe419a100 msgr2=0x7f1fe41a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:45.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.133+0000 7f1feb568640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fe419a100 0x7f1fe41a46b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00cad0 tx=0x7f1fdc00cf90 comp rx=0 tx=0).stop 2024-04-23T18:46:45.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.133+0000 7f1feb568640 1 -- 172.21.15.132:0/3087677539 shutdown_connections 2024-04-23T18:46:45.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.133+0000 7f1feb568640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f1fd00a3d20 0x7f1fd00a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.133+0000 7f1feb568640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fe419a100 0x7f1fe41a46b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.133+0000 7f1feb568640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fe4106170 0x7f1fe4199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.133+0000 7f1feb568640 1 --2- 172.21.15.132:0/3087677539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fe41010a0 0x7f1fe4199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.133+0000 7f1feb568640 1 -- 172.21.15.132:0/3087677539 >> 172.21.15.132:0/3087677539 conn(0x7f1fe4077eb0 msgr2=0x7f1fe4107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:45.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.133+0000 7f1feb568640 1 -- 172.21.15.132:0/3087677539 shutdown_connections 2024-04-23T18:46:45.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.133+0000 7f1feb568640 1 -- 172.21.15.132:0/3087677539 wait complete. 2024-04-23T18:46:45.135 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T18:46:45.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.243+0000 7f53d7ce1640 1 -- 172.21.15.132:0/3560385924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53d0107290 msgr2=0x7f53d010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:45.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.243+0000 7f53d7ce1640 1 --2- 172.21.15.132:0/3560385924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53d0107290 0x7f53d010ee30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009ca0 tx=0x7f53cc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.243+0000 7f53d7ce1640 1 -- 172.21.15.132:0/3560385924 shutdown_connections 2024-04-23T18:46:45.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.243+0000 7f53d7ce1640 1 --2- 172.21.15.132:0/3560385924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d0069d00 0x7f53d0106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.243+0000 7f53d7ce1640 1 --2- 172.21.15.132:0/3560385924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53d0107290 0x7f53d010ee30 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.243+0000 7f53d7ce1640 1 --2- 172.21.15.132:0/3560385924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d0069350 0x7f53d0069730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.243+0000 7f53d7ce1640 1 -- 172.21.15.132:0/3560385924 >> 172.21.15.132:0/3560385924 conn(0x7f53d0077cb0 msgr2=0x7f53d00780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d7ce1640 1 -- 172.21.15.132:0/3560385924 shutdown_connections 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d7ce1640 1 -- 172.21.15.132:0/3560385924 wait complete. 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d7ce1640 1 Processor -- start 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d7ce1640 1 -- start start 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d7ce1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d0069350 0x7f53d0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d7ce1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d0107290 0x7f53d0198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d7ce1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53d019cf70 0x7f53d019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d7ce1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0112200 con 0x7f53d019cf70 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d7ce1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0112080 con 0x7f53d0069350 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d7ce1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0112380 con 0x7f53d0107290 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d627e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53d019cf70 0x7f53d019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d627e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53d019cf70 0x7f53d019d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37846/0 (socket says 172.21.15.132:37846) 2024-04-23T18:46:45.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.244+0000 7f53d627e640 1 -- 172.21.15.132:0/424854961 learned_addr learned my addr 172.21.15.132:0/424854961 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53d527c640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d0107290 0x7f53d0198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53d5a7d640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d0069350 0x7f53d0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53d627e640 1 -- 172.21.15.132:0/424854961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d0107290 msgr2=0x7f53d0198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53d627e640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d0107290 0x7f53d0198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53d627e640 1 -- 172.21.15.132:0/424854961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d0069350 msgr2=0x7f53d0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53d627e640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d0069350 0x7f53d0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53d627e640 1 -- 172.21.15.132:0/424854961 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d019da60 con 0x7f53d019cf70 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53d627e640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53d019cf70 0x7f53d019d420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009430 tx=0x7f53cc0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53c6ffd640 1 -- 172.21.15.132:0/424854961 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc003c30 con 0x7f53d019cf70 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53c6ffd640 1 -- 172.21.15.132:0/424854961 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53cc003dd0 con 0x7f53d019cf70 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53d7ce1640 1 -- 172.21.15.132:0/424854961 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d019dcf0 con 0x7f53d019cf70 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53c6ffd640 1 -- 172.21.15.132:0/424854961 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc02faa0 con 0x7f53d019cf70 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.245+0000 7f53d7ce1640 1 -- 172.21.15.132:0/424854961 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d01a5510 con 0x7f53d019cf70 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.246+0000 7f53d7ce1640 1 -- 172.21.15.132:0/424854961 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d0068d70 con 0x7f53d019cf70 2024-04-23T18:46:45.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.247+0000 7f53c6ffd640 1 -- 172.21.15.132:0/424854961 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f53cc02fc40 con 0x7f53d019cf70 2024-04-23T18:46:45.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.248+0000 7f53c6ffd640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f53bc0a3d20 0x7f53bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:45.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.248+0000 7f53d5a7d640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f53bc0a3d20 0x7f53bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:45.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.248+0000 7f53c6ffd640 1 -- 172.21.15.132:0/424854961 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f53cc0f1db0 con 0x7f53d019cf70 2024-04-23T18:46:45.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.248+0000 7f53d5a7d640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f53bc0a3d20 0x7f53bc0a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f53c800bbe0 tx=0x7f53c800b390 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:45.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.251+0000 7f53c6ffd640 1 -- 172.21.15.132:0/424854961 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53cc0be520 con 0x7f53d019cf70 2024-04-23T18:46:45.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.439+0000 7f53d7ce1640 1 -- 172.21.15.132:0/424854961 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f53d0199a00 con 0x7f53d019cf70 2024-04-23T18:46:45.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.439+0000 7f53c6ffd640 1 -- 172.21.15.132:0/424854961 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v99) ==== 81+0+957 (secure 0 0 0) 0x7f53d0199a00 con 0x7f53d019cf70 2024-04-23T18:46:45.431 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/2124409853 2024-04-24T18:36:02.717273+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/3095429281 2024-04-24T18:36:02.717273+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/899407689 2024-04-24T18:36:02.717273+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/4132015592 2024-04-24T18:26:54.042392+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:6801/636352941 2024-04-24T18:36:02.717273+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:6801/532999776 2024-04-24T18:26:54.042392+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/1857586348 2024-04-24T18:36:02.717273+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/1778322727 2024-04-24T18:26:54.042392+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/1210637601 2024-04-24T18:26:27.197140+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:6800/532999776 2024-04-24T18:26:54.042392+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:6801/2811136616 2024-04-24T18:26:27.197140+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/2746316554 2024-04-24T18:26:27.197140+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/3819980873 2024-04-24T18:26:27.197140+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:6800/636352941 2024-04-24T18:36:02.717273+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:6800/2811136616 2024-04-24T18:26:27.197140+0000 2024-04-23T18:46:45.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/1096830498 2024-04-24T18:26:54.042392+0000 2024-04-23T18:46:45.433 INFO:teuthology.orchestra.run.smithi132.stderr:listed 16 entries 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.441+0000 7f53d7ce1640 1 -- 172.21.15.132:0/424854961 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f53bc0a3d20 msgr2=0x7f53bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.441+0000 7f53d7ce1640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f53bc0a3d20 0x7f53bc0a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f53c800bbe0 tx=0x7f53c800b390 comp rx=0 tx=0).stop 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.441+0000 7f53d7ce1640 1 -- 172.21.15.132:0/424854961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53d019cf70 msgr2=0x7f53d019d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.441+0000 7f53d7ce1640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53d019cf70 0x7f53d019d420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009430 tx=0x7f53cc0319f0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.441+0000 7f53d7ce1640 1 -- 172.21.15.132:0/424854961 shutdown_connections 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.441+0000 7f53d7ce1640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f53bc0a3d20 0x7f53bc0a61e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.441+0000 7f53d7ce1640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53d0107290 0x7f53d0198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.441+0000 7f53d7ce1640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53d019cf70 0x7f53d019d420 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.441+0000 7f53d7ce1640 1 --2- 172.21.15.132:0/424854961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d0069350 0x7f53d0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.441+0000 7f53d7ce1640 1 -- 172.21.15.132:0/424854961 >> 172.21.15.132:0/424854961 conn(0x7f53d0077cb0 msgr2=0x7f53d00ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.441+0000 7f53d7ce1640 1 -- 172.21.15.132:0/424854961 shutdown_connections 2024-04-23T18:46:45.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.441+0000 7f53d7ce1640 1 -- 172.21.15.132:0/424854961 wait complete. 2024-04-23T18:46:45.445 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-23T18:46:45.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:46:45.445 DEBUG:teuthology.orchestra.run.smithi132:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-23T18:46:45.486 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2024-04-23T18:46:45.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:46:45.486 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T18:46:45.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:46:45.557 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-23T18:46:45.612 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T18:46:45.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:46:45.612 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T18:46:45.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:46:45.676 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-04-23T18:46:45.779 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-23T18:46:45.791 INFO:tasks.cephadm:Teardown begin 2024-04-23T18:46:45.791 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T18:46:45.791 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:46:45.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.900+0000 7f4037205640 1 -- 172.21.15.132:0/903584164 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4030106160 msgr2=0x7f403010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:45.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.900+0000 7f4037205640 1 --2- 172.21.15.132:0/903584164 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4030106160 0x7f403010dd30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4028009ca0 tx=0x7f402802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.900+0000 7f4037205640 1 -- 172.21.15.132:0/903584164 shutdown_connections 2024-04-23T18:46:45.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.900+0000 7f4037205640 1 --2- 172.21.15.132:0/903584164 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4030106160 0x7f403010dd30 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.900+0000 7f4037205640 1 --2- 172.21.15.132:0/903584164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4030101a40 0x7f4030105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.900+0000 7f4037205640 1 --2- 172.21.15.132:0/903584164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4030101090 0x7f4030101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.900+0000 7f4037205640 1 -- 172.21.15.132:0/903584164 >> 172.21.15.132:0/903584164 conn(0x7f40300771d0 msgr2=0x7f40300775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:45.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.900+0000 7f4037205640 1 -- 172.21.15.132:0/903584164 shutdown_connections 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.900+0000 7f4037205640 1 -- 172.21.15.132:0/903584164 wait complete. 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.900+0000 7f4037205640 1 Processor -- start 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.900+0000 7f4037205640 1 -- start start 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f4037205640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4030101090 0x7f4030197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f4037205640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4030106160 0x7f4030197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f4037205640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f403019bf10 0x7f403019c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f4037205640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f40301110d0 con 0x7f403019bf10 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f4037205640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4030110f50 con 0x7f4030101090 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f4037205640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4030111250 con 0x7f4030106160 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f40357a2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f403019bf10 0x7f403019c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f40357a2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f403019bf10 0x7f403019c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37864/0 (socket says 172.21.15.132:37864) 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f40357a2640 1 -- 172.21.15.132:0/1795119841 learned_addr learned my addr 172.21.15.132:0/1795119841 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f4034fa1640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4030101090 0x7f4030197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:45.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f402ffff640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4030106160 0x7f4030197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:45.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f40357a2640 1 -- 172.21.15.132:0/1795119841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4030106160 msgr2=0x7f4030197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:45.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f40357a2640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4030106160 0x7f4030197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f40357a2640 1 -- 172.21.15.132:0/1795119841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4030101090 msgr2=0x7f4030197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:45.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f40357a2640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4030101090 0x7f4030197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:45.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f40357a2640 1 -- 172.21.15.132:0/1795119841 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403019ca00 con 0x7f403019bf10 2024-04-23T18:46:45.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f40357a2640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f403019bf10 0x7f403019c3c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4028005cc0 tx=0x7f402802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:45.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f402dffb640 1 -- 172.21.15.132:0/1795119841 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028003bf0 con 0x7f403019bf10 2024-04-23T18:46:45.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f4037205640 1 -- 172.21.15.132:0/1795119841 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403019cc90 con 0x7f403019bf10 2024-04-23T18:46:45.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f402dffb640 1 -- 172.21.15.132:0/1795119841 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4028003d90 con 0x7f403019bf10 2024-04-23T18:46:45.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.901+0000 7f402dffb640 1 -- 172.21.15.132:0/1795119841 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402802fbc0 con 0x7f403019bf10 2024-04-23T18:46:45.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.902+0000 7f4037205640 1 -- 172.21.15.132:0/1795119841 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40301a44b0 con 0x7f403019bf10 2024-04-23T18:46:45.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.903+0000 7f4037205640 1 -- 172.21.15.132:0/1795119841 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4030102560 con 0x7f403019bf10 2024-04-23T18:46:45.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.904+0000 7f402dffb640 1 -- 172.21.15.132:0/1795119841 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f402802fd60 con 0x7f403019bf10 2024-04-23T18:46:45.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.904+0000 7f402dffb640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f40200a3d20 0x7f40200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:45.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.904+0000 7f4034fa1640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f40200a3d20 0x7f40200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:45.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.904+0000 7f402dffb640 1 -- 172.21.15.132:0/1795119841 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f40280f1e50 con 0x7f403019bf10 2024-04-23T18:46:45.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.905+0000 7f4034fa1640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f40200a3d20 0x7f40200a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40240058c0 tx=0x7f402400a250 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:45.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:45.909+0000 7f402dffb640 1 -- 172.21.15.132:0/1795119841 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4028004610 con 0x7f403019bf10 2024-04-23T18:46:46.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:45 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2656798858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:46:46.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:45 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/3087677539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:46:46.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:45 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/424854961' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T18:46:46.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:45 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2656798858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:46:46.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:45 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/3087677539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:46:46.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:45 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/424854961' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T18:46:46.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:45 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2656798858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:46:46.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:45 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/3087677539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:46:46.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:45 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/424854961' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T18:46:46.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.128+0000 7f4037205640 1 -- 172.21.15.132:0/1795119841 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40301989a0 con 0x7f403019bf10 2024-04-23T18:46:46.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.129+0000 7f402dffb640 1 -- 172.21.15.132:0/1795119841 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11946 (secure 0 0 0) 0x7f40280098b0 con 0x7f403019bf10 2024-04-23T18:46:46.121 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:46:46.121 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:39:35.850746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24460,24425,24394,24491,24515]},"id":1}]} 2024-04-23T18:46:46.121 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 30 2024-04-23T18:46:46.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.131+0000 7f4037205640 1 -- 172.21.15.132:0/1795119841 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f40200a3d20 msgr2=0x7f40200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.131+0000 7f4037205640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f40200a3d20 0x7f40200a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40240058c0 tx=0x7f402400a250 comp rx=0 tx=0).stop 2024-04-23T18:46:46.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.131+0000 7f4037205640 1 -- 172.21.15.132:0/1795119841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f403019bf10 msgr2=0x7f403019c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.131+0000 7f4037205640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f403019bf10 0x7f403019c3c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4028005cc0 tx=0x7f402802f380 comp rx=0 tx=0).stop 2024-04-23T18:46:46.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.131+0000 7f4037205640 1 -- 172.21.15.132:0/1795119841 shutdown_connections 2024-04-23T18:46:46.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.131+0000 7f4037205640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f40200a3d20 0x7f40200a61e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.131+0000 7f4037205640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4030106160 0x7f4030197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.131+0000 7f4037205640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f403019bf10 0x7f403019c3c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.131+0000 7f4037205640 1 --2- 172.21.15.132:0/1795119841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4030101090 0x7f4030197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.131+0000 7f4037205640 1 -- 172.21.15.132:0/1795119841 >> 172.21.15.132:0/1795119841 conn(0x7f40300771d0 msgr2=0x7f40300fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:46.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.131+0000 7f4037205640 1 -- 172.21.15.132:0/1795119841 shutdown_connections 2024-04-23T18:46:46.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.131+0000 7f4037205640 1 -- 172.21.15.132:0/1795119841 wait complete. 2024-04-23T18:46:46.134 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:46:46.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.240+0000 7fe55d144640 1 -- 172.21.15.132:0/3226944765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe558106170 msgr2=0x7fe55810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.240+0000 7fe55d144640 1 --2- 172.21.15.132:0/3226944765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe558106170 0x7fe55810dd40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe550009ca0 tx=0x7fe55002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.240+0000 7fe55d144640 1 -- 172.21.15.132:0/3226944765 shutdown_connections 2024-04-23T18:46:46.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.240+0000 7fe55d144640 1 --2- 172.21.15.132:0/3226944765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe558106170 0x7fe55810dd40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.240+0000 7fe55d144640 1 --2- 172.21.15.132:0/3226944765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe558101a50 0x7fe558105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.240+0000 7fe55d144640 1 --2- 172.21.15.132:0/3226944765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5581010a0 0x7fe558101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.240+0000 7fe55d144640 1 -- 172.21.15.132:0/3226944765 >> 172.21.15.132:0/3226944765 conn(0x7fe558077f50 msgr2=0x7fe558078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:46.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.240+0000 7fe55d144640 1 -- 172.21.15.132:0/3226944765 shutdown_connections 2024-04-23T18:46:46.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.240+0000 7fe55d144640 1 -- 172.21.15.132:0/3226944765 wait complete. 2024-04-23T18:46:46.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.241+0000 7fe55d144640 1 Processor -- start 2024-04-23T18:46:46.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.241+0000 7fe55d144640 1 -- start start 2024-04-23T18:46:46.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.241+0000 7fe55d144640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5581010a0 0x7fe558198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:46.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.241+0000 7fe55d144640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe558106170 0x7fe558198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:46.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.241+0000 7fe55d144640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe55819cf20 0x7fe55819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:46.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.241+0000 7fe55d144640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581110c0 con 0x7fe558106170 2024-04-23T18:46:46.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.241+0000 7fe55d144640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe558110f40 con 0x7fe5581010a0 2024-04-23T18:46:46.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.241+0000 7fe55d144640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe558111240 con 0x7fe55819cf20 2024-04-23T18:46:46.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.241+0000 7fe55659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe558106170 0x7fe558198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:46.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe55659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe558106170 0x7fe558198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37868/0 (socket says 172.21.15.132:37868) 2024-04-23T18:46:46.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe55659c640 1 -- 172.21.15.132:0/2150363333 learned_addr learned my addr 172.21.15.132:0/2150363333 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:46.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe556d9d640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5581010a0 0x7fe558198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:46.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe55759e640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe55819cf20 0x7fe55819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:46.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe55659c640 1 -- 172.21.15.132:0/2150363333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe55819cf20 msgr2=0x7fe55819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe55659c640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe55819cf20 0x7fe55819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe55659c640 1 -- 172.21.15.132:0/2150363333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5581010a0 msgr2=0x7fe558198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe55659c640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5581010a0 0x7fe558198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe55659c640 1 -- 172.21.15.132:0/2150363333 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55819da10 con 0x7fe558106170 2024-04-23T18:46:46.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe55659c640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe558106170 0x7fe558198ba0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe54800c970 tx=0x7fe54800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:46.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe54ffff640 1 -- 172.21.15.132:0/2150363333 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548007bb0 con 0x7fe558106170 2024-04-23T18:46:46.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe55d144640 1 -- 172.21.15.132:0/2150363333 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55819dd00 con 0x7fe558106170 2024-04-23T18:46:46.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe54ffff640 1 -- 172.21.15.132:0/2150363333 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe548007d50 con 0x7fe558106170 2024-04-23T18:46:46.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe54ffff640 1 -- 172.21.15.132:0/2150363333 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548004d80 con 0x7fe558106170 2024-04-23T18:46:46.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.242+0000 7fe55d144640 1 -- 172.21.15.132:0/2150363333 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5581a54f0 con 0x7fe558106170 2024-04-23T18:46:46.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.243+0000 7fe55d144640 1 -- 172.21.15.132:0/2150363333 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe558068d70 con 0x7fe558106170 2024-04-23T18:46:46.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.244+0000 7fe54ffff640 1 -- 172.21.15.132:0/2150363333 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fe548004f60 con 0x7fe558106170 2024-04-23T18:46:46.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.245+0000 7fe54ffff640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe5440a3d20 0x7fe5440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:46.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.245+0000 7fe556d9d640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe5440a3d20 0x7fe5440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:46.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.245+0000 7fe54ffff640 1 -- 172.21.15.132:0/2150363333 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7fe5480d0500 con 0x7fe558106170 2024-04-23T18:46:46.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.245+0000 7fe556d9d640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe5440a3d20 0x7fe5440a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe54000abe0 tx=0x7fe54000a2d0 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:46.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.250+0000 7fe54ffff640 1 -- 172.21.15.132:0/2150363333 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe548014030 con 0x7fe558106170 2024-04-23T18:46:46.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.463+0000 7fe55d144640 1 -- 172.21.15.132:0/2150363333 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe558101a50 con 0x7fe558106170 2024-04-23T18:46:46.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.464+0000 7fe54ffff640 1 -- 172.21.15.132:0/2150363333 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11946 (secure 0 0 0) 0x7fe54809cb60 con 0x7fe558106170 2024-04-23T18:46:46.456 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:46:46.457 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/152253190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":152253190},{"type":"v1","addr":"172.21.15.132:6835","nonce":152253190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1545569950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1545569950},{"type":"v1","addr":"172.21.15.132:6837","nonce":1545569950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:34:00.571339+0000","modified":"2024-04-23T18:39:35.850746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24460,"mds_3":24425,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6839/579251026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":579251026},{"type":"v1","addr":"172.21.15.132:6839","nonce":579251026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6841/826931131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":826931131},{"type":"v1","addr":"172.21.15.132:6841","nonce":826931131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6833/1034092547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1034092547},{"type":"v1","addr":"172.21.15.157:6833","nonce":1034092547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6833/3556073608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3556073608},{"type":"v1","addr":"172.21.15.148:6833","nonce":3556073608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4121037781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4121037781},{"type":"v1","addr":"172.21.15.157:6835","nonce":4121037781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/2670322328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2670322328},{"type":"v1","addr":"172.21.15.157:6837","nonce":2670322328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/397710093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":397710093},{"type":"v1","addr":"172.21.15.148:6835","nonce":397710093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1528717147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1528717147},{"type":"v1","addr":"172.21.15.157:6839","nonce":1528717147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2626530887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2626530887},{"type":"v1","addr":"172.21.15.148:6837","nonce":2626530887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/4115412726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":4115412726},{"type":"v1","addr":"172.21.15.148:6839","nonce":4115412726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24460,24425,24394,24491,24515]},"id":1}]} 2024-04-23T18:46:46.457 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 30 2024-04-23T18:46:46.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.466+0000 7fe55d144640 1 -- 172.21.15.132:0/2150363333 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe5440a3d20 msgr2=0x7fe5440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.466+0000 7fe55d144640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe5440a3d20 0x7fe5440a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe54000abe0 tx=0x7fe54000a2d0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.466+0000 7fe55d144640 1 -- 172.21.15.132:0/2150363333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe558106170 msgr2=0x7fe558198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.466+0000 7fe55d144640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe558106170 0x7fe558198ba0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe54800c970 tx=0x7fe54800cd70 comp rx=0 tx=0).stop 2024-04-23T18:46:46.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.466+0000 7fe55d144640 1 -- 172.21.15.132:0/2150363333 shutdown_connections 2024-04-23T18:46:46.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.466+0000 7fe55d144640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fe5440a3d20 0x7fe5440a61e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.466+0000 7fe55d144640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe55819cf20 0x7fe55819d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.466+0000 7fe55d144640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe558106170 0x7fe558198ba0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.466+0000 7fe55d144640 1 --2- 172.21.15.132:0/2150363333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5581010a0 0x7fe558198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.466+0000 7fe55d144640 1 -- 172.21.15.132:0/2150363333 >> 172.21.15.132:0/2150363333 conn(0x7fe558077f50 msgr2=0x7fe5580693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:46.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.466+0000 7fe55d144640 1 -- 172.21.15.132:0/2150363333 shutdown_connections 2024-04-23T18:46:46.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.466+0000 7fe55d144640 1 -- 172.21.15.132:0/2150363333 wait complete. 2024-04-23T18:46:46.470 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:46:46.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.575+0000 7f8bccc8a640 1 -- 172.21.15.132:0/2256524958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bc8101a40 msgr2=0x7f8bc8105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.575+0000 7f8bccc8a640 1 --2- 172.21.15.132:0/2256524958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bc8101a40 0x7f8bc8105a80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009900 tx=0x7f8bb002a410 comp rx=0 tx=0).stop 2024-04-23T18:46:46.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.575+0000 7f8bccc8a640 1 -- 172.21.15.132:0/2256524958 shutdown_connections 2024-04-23T18:46:46.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.575+0000 7f8bccc8a640 1 --2- 172.21.15.132:0/2256524958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8106160 0x7f8bc810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.575+0000 7f8bccc8a640 1 --2- 172.21.15.132:0/2256524958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bc8101a40 0x7f8bc8105a80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.575+0000 7f8bccc8a640 1 --2- 172.21.15.132:0/2256524958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bc8101090 0x7f8bc8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.575+0000 7f8bccc8a640 1 -- 172.21.15.132:0/2256524958 >> 172.21.15.132:0/2256524958 conn(0x7f8bc80771d0 msgr2=0x7f8bc80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:46.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.575+0000 7f8bccc8a640 1 -- 172.21.15.132:0/2256524958 shutdown_connections 2024-04-23T18:46:46.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.575+0000 7f8bccc8a640 1 -- 172.21.15.132:0/2256524958 wait complete. 2024-04-23T18:46:46.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bccc8a640 1 Processor -- start 2024-04-23T18:46:46.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bccc8a640 1 -- start start 2024-04-23T18:46:46.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bccc8a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bc8101090 0x7f8bc81995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bccc8a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8106160 0x7f8bc8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bccc8a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bc819a070 0x7f8bc81a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bccc8a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8111080 con 0x7f8bc8101090 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bccc8a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8110f00 con 0x7f8bc8106160 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bccc8a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8111200 con 0x7f8bc819a070 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bc659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bc8101090 0x7f8bc81995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bc659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bc8101090 0x7f8bc81995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37884/0 (socket says 172.21.15.132:37884) 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bc659c640 1 -- 172.21.15.132:0/756449642 learned_addr learned my addr 172.21.15.132:0/756449642 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bc659c640 1 -- 172.21.15.132:0/756449642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bc819a070 msgr2=0x7f8bc81a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bc5d9b640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8106160 0x7f8bc8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bc659c640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bc819a070 0x7f8bc81a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bc659c640 1 -- 172.21.15.132:0/756449642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8106160 msgr2=0x7f8bc8199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bc659c640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8106160 0x7f8bc8199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bc659c640 1 -- 172.21.15.132:0/756449642 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc81a4e20 con 0x7f8bc8101090 2024-04-23T18:46:46.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bc5d9b640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8106160 0x7f8bc8199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:46:46.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.576+0000 7f8bc659c640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bc8101090 0x7f8bc81995a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800d950 tx=0x7f8bb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:46.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.577+0000 7f8bbf7fe640 1 -- 172.21.15.132:0/756449642 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8017070 con 0x7f8bc8101090 2024-04-23T18:46:46.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.577+0000 7f8bbf7fe640 1 -- 172.21.15.132:0/756449642 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb8009c30 con 0x7f8bc8101090 2024-04-23T18:46:46.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.577+0000 7f8bccc8a640 1 -- 172.21.15.132:0/756449642 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc81a5110 con 0x7f8bc8101090 2024-04-23T18:46:46.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.577+0000 7f8bbf7fe640 1 -- 172.21.15.132:0/756449642 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb800f620 con 0x7f8bc8101090 2024-04-23T18:46:46.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.577+0000 7f8bccc8a640 1 -- 172.21.15.132:0/756449642 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc81a5660 con 0x7f8bc8101090 2024-04-23T18:46:46.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.578+0000 7f8bccc8a640 1 -- 172.21.15.132:0/756449642 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc80691c0 con 0x7f8bc8101090 2024-04-23T18:46:46.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.579+0000 7f8bbf7fe640 1 -- 172.21.15.132:0/756449642 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f8bb8010460 con 0x7f8bc8101090 2024-04-23T18:46:46.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.579+0000 7f8bbf7fe640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8bb40a3d20 0x7f8bb40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:46.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.579+0000 7f8bc5d9b640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8bb40a3d20 0x7f8bb40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:46.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.579+0000 7f8bbf7fe640 1 -- 172.21.15.132:0/756449642 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f8bb80d0530 con 0x7f8bc8101090 2024-04-23T18:46:46.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.580+0000 7f8bc5d9b640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8bb40a3d20 0x7f8bb40a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0000c00 tx=0x7f8bb0031400 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:46.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.584+0000 7f8bbf7fe640 1 -- 172.21.15.132:0/756449642 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8bb809cb90 con 0x7f8bc8101090 2024-04-23T18:46:46.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.763+0000 7f8bccc8a640 1 -- 172.21.15.132:0/756449642 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8bc806ac20 con 0x7f8bc8101090 2024-04-23T18:46:46.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.765+0000 7f8bbf7fe640 1 -- 172.21.15.132:0/756449642 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23318 (secure 0 0 0) 0x7f8bb8009de0 con 0x7f8bc8101090 2024-04-23T18:46:46.757 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-23T18:46:46.758 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":99,"fsid":"c34642b4-019e-11ef-bc93-c7b262605968","created":"2024-04-23T18:26:00.784473+0000","modified":"2024-04-23T18:46:20.673047+0000","last_up_change":"2024-04-23T18:33:10.814251+0000","last_in_change":"2024-04-23T18:32:50.857709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:29:20.132451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:33:58.407521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:33:59.518206+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5387e5b6-e309-4bf9-abcd-bd917b437059","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6803","nonce":1754141150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6805","nonce":1754141150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6809","nonce":1754141150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1754141150},{"type":"v1","addr":"172.21.15.132:6807","nonce":1754141150}]},"public_addr":"172.21.15.132:6803/1754141150","cluster_addr":"172.21.15.132:6805/1754141150","heartbeat_back_addr":"172.21.15.132:6809/1754141150","heartbeat_front_addr":"172.21.15.132:6807/1754141150","state":["exists","up"]},{"osd":1,"uuid":"962fbcd8-143f-4f15-9e83-6afed0c5a72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6801","nonce":2532595181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6803","nonce":2532595181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6807","nonce":2532595181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2532595181},{"type":"v1","addr":"172.21.15.148:6805","nonce":2532595181}]},"public_addr":"172.21.15.148:6801/2532595181","cluster_addr":"172.21.15.148:6803/2532595181","heartbeat_back_addr":"172.21.15.148:6807/2532595181","heartbeat_front_addr":"172.21.15.148:6805/2532595181","state":["exists","up"]},{"osd":2,"uuid":"3c4885a5-c35c-4b55-bbd2-9370fc797096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6801","nonce":1299586857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6803","nonce":1299586857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6807","nonce":1299586857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1299586857},{"type":"v1","addr":"172.21.15.157:6805","nonce":1299586857}]},"public_addr":"172.21.15.157:6801/1299586857","cluster_addr":"172.21.15.157:6803/1299586857","heartbeat_back_addr":"172.21.15.157:6807/1299586857","heartbeat_front_addr":"172.21.15.157:6805/1299586857","state":["exists","up"]},{"osd":3,"uuid":"de1bb55b-fced-40bd-b2c3-81a227b2e891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6811","nonce":2500771155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6813","nonce":2500771155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500771155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2500771155},{"type":"v1","addr":"172.21.15.132:6815","nonce":2500771155}]},"public_addr":"172.21.15.132:6811/2500771155","cluster_addr":"172.21.15.132:6813/2500771155","heartbeat_back_addr":"172.21.15.132:6817/2500771155","heartbeat_front_addr":"172.21.15.132:6815/2500771155","state":["exists","up"]},{"osd":4,"uuid":"0e84996c-3580-48d6-b38f-ad1cd3217e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6809","nonce":1930596121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6811","nonce":1930596121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6815","nonce":1930596121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1930596121},{"type":"v1","addr":"172.21.15.148:6813","nonce":1930596121}]},"public_addr":"172.21.15.148:6809/1930596121","cluster_addr":"172.21.15.148:6811/1930596121","heartbeat_back_addr":"172.21.15.148:6815/1930596121","heartbeat_front_addr":"172.21.15.148:6813/1930596121","state":["exists","up"]},{"osd":5,"uuid":"f06f9283-8099-4922-8e1c-f28c4ea57a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6809","nonce":3700977653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6811","nonce":3700977653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6815","nonce":3700977653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3700977653},{"type":"v1","addr":"172.21.15.157:6813","nonce":3700977653}]},"public_addr":"172.21.15.157:6809/3700977653","cluster_addr":"172.21.15.157:6811/3700977653","heartbeat_back_addr":"172.21.15.157:6815/3700977653","heartbeat_front_addr":"172.21.15.157:6813/3700977653","state":["exists","up"]},{"osd":6,"uuid":"3b1233e4-b273-4c3e-89a4-ecec0758d4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6819","nonce":3702092226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6821","nonce":3702092226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6825","nonce":3702092226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":3702092226},{"type":"v1","addr":"172.21.15.132:6823","nonce":3702092226}]},"public_addr":"172.21.15.132:6819/3702092226","cluster_addr":"172.21.15.132:6821/3702092226","heartbeat_back_addr":"172.21.15.132:6825/3702092226","heartbeat_front_addr":"172.21.15.132:6823/3702092226","state":["exists","up"]},{"osd":7,"uuid":"93425a43-d24f-4456-81d6-239c59996a22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6817","nonce":199627413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6819","nonce":199627413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6823","nonce":199627413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":199627413},{"type":"v1","addr":"172.21.15.148:6821","nonce":199627413}]},"public_addr":"172.21.15.148:6817/199627413","cluster_addr":"172.21.15.148:6819/199627413","heartbeat_back_addr":"172.21.15.148:6823/199627413","heartbeat_front_addr":"172.21.15.148:6821/199627413","state":["exists","up"]},{"osd":8,"uuid":"f9285b86-b921-4e03-af87-4fd814b866cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6817","nonce":3005625159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6819","nonce":3005625159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6823","nonce":3005625159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3005625159},{"type":"v1","addr":"172.21.15.157:6821","nonce":3005625159}]},"public_addr":"172.21.15.157:6817/3005625159","cluster_addr":"172.21.15.157:6819/3005625159","heartbeat_back_addr":"172.21.15.157:6823/3005625159","heartbeat_front_addr":"172.21.15.157:6821/3005625159","state":["exists","up"]},{"osd":9,"uuid":"90c3d0c2-e221-41ac-8da7-b272837f6283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6827","nonce":3598365445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6829","nonce":3598365445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6833","nonce":3598365445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3598365445},{"type":"v1","addr":"172.21.15.132:6831","nonce":3598365445}]},"public_addr":"172.21.15.132:6827/3598365445","cluster_addr":"172.21.15.132:6829/3598365445","heartbeat_back_addr":"172.21.15.132:6833/3598365445","heartbeat_front_addr":"172.21.15.132:6831/3598365445","state":["exists","up"]},{"osd":10,"uuid":"e548975f-da5d-4394-a5c6-e658a6c89046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6825","nonce":2494142299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6827","nonce":2494142299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6831","nonce":2494142299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2494142299},{"type":"v1","addr":"172.21.15.148:6829","nonce":2494142299}]},"public_addr":"172.21.15.148:6825/2494142299","cluster_addr":"172.21.15.148:6827/2494142299","heartbeat_back_addr":"172.21.15.148:6831/2494142299","heartbeat_front_addr":"172.21.15.148:6829/2494142299","state":["exists","up"]},{"osd":11,"uuid":"5d9c5bae-d624-411c-8636-8156934d51fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6825","nonce":2341045818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6827","nonce":2341045818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6831","nonce":2341045818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2341045818},{"type":"v1","addr":"172.21.15.157:6829","nonce":2341045818}]},"public_addr":"172.21.15.157:6825/2341045818","cluster_addr":"172.21.15.157:6827/2341045818","heartbeat_back_addr":"172.21.15.157:6831/2341045818","heartbeat_front_addr":"172.21.15.157:6829/2341045818","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:29.098037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:28:53.526360+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:17.498493+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:29:42.694890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:07.512668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:32.429699+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:30:58.068599+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:24.582298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:31:50.120339+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:16.505641+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:32:42.810421+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:33:09.492028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/2124409853":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/3095429281":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/899407689":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/4132015592":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6801/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1857586348":"2024-04-24T18:36:02.717273+0000","172.21.15.132:0/1778322727":"2024-04-24T18:26:54.042392+0000","172.21.15.132:0/1210637601":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/532999776":"2024-04-24T18:26:54.042392+0000","172.21.15.132:6801/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/2746316554":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/3819980873":"2024-04-24T18:26:27.197140+0000","172.21.15.132:6800/636352941":"2024-04-24T18:36:02.717273+0000","172.21.15.132:6800/2811136616":"2024-04-24T18:26:27.197140+0000","172.21.15.132:0/1096830498":"2024-04-24T18:26:54.042392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:46:46.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.767+0000 7f8bccc8a640 1 -- 172.21.15.132:0/756449642 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8bb40a3d20 msgr2=0x7f8bb40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.767+0000 7f8bccc8a640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8bb40a3d20 0x7f8bb40a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0000c00 tx=0x7f8bb0031400 comp rx=0 tx=0).stop 2024-04-23T18:46:46.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.767+0000 7f8bccc8a640 1 -- 172.21.15.132:0/756449642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bc8101090 msgr2=0x7f8bc81995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.767+0000 7f8bccc8a640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bc8101090 0x7f8bc81995a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800d950 tx=0x7f8bb800de10 comp rx=0 tx=0).stop 2024-04-23T18:46:46.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.767+0000 7f8bccc8a640 1 -- 172.21.15.132:0/756449642 shutdown_connections 2024-04-23T18:46:46.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.767+0000 7f8bccc8a640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f8bb40a3d20 0x7f8bb40a61e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.767+0000 7f8bccc8a640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bc819a070 0x7f8bc81a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.767+0000 7f8bccc8a640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8106160 0x7f8bc8199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.767+0000 7f8bccc8a640 1 --2- 172.21.15.132:0/756449642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bc8101090 0x7f8bc81995a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.767+0000 7f8bccc8a640 1 -- 172.21.15.132:0/756449642 >> 172.21.15.132:0/756449642 conn(0x7f8bc80771d0 msgr2=0x7f8bc80feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:46.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.767+0000 7f8bccc8a640 1 -- 172.21.15.132:0/756449642 shutdown_connections 2024-04-23T18:46:46.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.767+0000 7f8bccc8a640 1 -- 172.21.15.132:0/756449642 wait complete. 2024-04-23T18:46:46.769 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:46 smithi132 ceph-mon[29178]: pgmap v332: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:46:46.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:46 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1795119841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:46:46.770 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:46 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2150363333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:46:46.771 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T18:46:46.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.878+0000 7f833bea3640 1 -- 172.21.15.132:0/3845069840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334105ee0 msgr2=0x7f83341062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.878+0000 7f833bea3640 1 --2- 172.21.15.132:0/3845069840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334105ee0 0x7f83341062c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8330009900 tx=0x7f833002a400 comp rx=0 tx=0).stop 2024-04-23T18:46:46.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.878+0000 7f833bea3640 1 -- 172.21.15.132:0/3845069840 shutdown_connections 2024-04-23T18:46:46.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.878+0000 7f833bea3640 1 --2- 172.21.15.132:0/3845069840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8334101b60 0x7f833410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.878+0000 7f833bea3640 1 --2- 172.21.15.132:0/3845069840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83341011a0 0x7f8334101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.878+0000 7f833bea3640 1 --2- 172.21.15.132:0/3845069840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334105ee0 0x7f83341062c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.878+0000 7f833bea3640 1 -- 172.21.15.132:0/3845069840 >> 172.21.15.132:0/3845069840 conn(0x7f8334076c10 msgr2=0x7f8334077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.878+0000 7f833bea3640 1 -- 172.21.15.132:0/3845069840 shutdown_connections 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.878+0000 7f833bea3640 1 -- 172.21.15.132:0/3845069840 wait complete. 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.878+0000 7f833bea3640 1 Processor -- start 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.878+0000 7f833bea3640 1 -- start start 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833bea3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83341011a0 0x7f833419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833bea3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334101b60 0x7f833419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833bea3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334105ee0 0x7f83341985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833bea3640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f83341110f0 con 0x7f8334105ee0 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833bea3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8334110f70 con 0x7f8334101b60 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833bea3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8334111270 con 0x7f83341011a0 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833a440640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334105ee0 0x7f83341985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833a440640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334105ee0 0x7f83341985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37902/0 (socket says 172.21.15.132:37902) 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833a440640 1 -- 172.21.15.132:0/56624009 learned_addr learned my addr 172.21.15.132:0/56624009 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:46.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f8339c3f640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83341011a0 0x7f833419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833943e640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334101b60 0x7f833419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833a440640 1 -- 172.21.15.132:0/56624009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83341011a0 msgr2=0x7f833419e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833a440640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83341011a0 0x7f833419e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833a440640 1 -- 172.21.15.132:0/56624009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334101b60 msgr2=0x7f833419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833a440640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334101b60 0x7f833419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833a440640 1 -- 172.21.15.132:0/56624009 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8334198d40 con 0x7f8334105ee0 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833943e640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334101b60 0x7f833419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833a440640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334105ee0 0x7f83341985c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f832c00cad0 tx=0x7f832c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f832affd640 1 -- 172.21.15.132:0/56624009 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c00ae20 con 0x7f8334105ee0 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833bea3640 1 -- 172.21.15.132:0/56624009 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8334199030 con 0x7f8334105ee0 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f832affd640 1 -- 172.21.15.132:0/56624009 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832c005020 con 0x7f8334105ee0 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f832affd640 1 -- 172.21.15.132:0/56624009 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c00f6b0 con 0x7f8334105ee0 2024-04-23T18:46:46.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.879+0000 7f833bea3640 1 -- 172.21.15.132:0/56624009 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83341a52d0 con 0x7f8334105ee0 2024-04-23T18:46:46.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.880+0000 7f833bea3640 1 -- 172.21.15.132:0/56624009 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8334102570 con 0x7f8334105ee0 2024-04-23T18:46:46.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.882+0000 7f832affd640 1 -- 172.21.15.132:0/56624009 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f832c0051d0 con 0x7f8334105ee0 2024-04-23T18:46:46.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.882+0000 7f832affd640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f83200a3d20 0x7f83200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:46.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.882+0000 7f8339c3f640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f83200a3d20 0x7f83200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:46.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.882+0000 7f832affd640 1 -- 172.21.15.132:0/56624009 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f832c0d0ba0 con 0x7f8334105ee0 2024-04-23T18:46:46.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.883+0000 7f8339c3f640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f83200a3d20 0x7f83200a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8330031440 tx=0x7f8330005e80 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:46.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:46.886+0000 7f832affd640 1 -- 172.21.15.132:0/56624009 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f832c014030 con 0x7f8334105ee0 2024-04-23T18:46:47.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:46 smithi148 ceph-mon[32311]: pgmap v332: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:46:47.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:46 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1795119841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:46:47.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:46 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2150363333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:46:47.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:46 smithi157 ceph-mon[32580]: pgmap v332: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-23T18:46:47.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:46 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1795119841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:46:47.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:46 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2150363333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:46:47.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.099+0000 7f833bea3640 1 -- 172.21.15.132:0/56624009 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f83340691c0 con 0x7f8334105ee0 2024-04-23T18:46:47.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.101+0000 7f832affd640 1 -- 172.21.15.132:0/56624009 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7f83340691c0 con 0x7f8334105ee0 2024-04-23T18:46:47.093 INFO:teuthology.orchestra.run.smithi132.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T18:46:47.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.105+0000 7f833bea3640 1 -- 172.21.15.132:0/56624009 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f83200a3d20 msgr2=0x7f83200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:47.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.105+0000 7f833bea3640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f83200a3d20 0x7f83200a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8330031440 tx=0x7f8330005e80 comp rx=0 tx=0).stop 2024-04-23T18:46:47.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.105+0000 7f833bea3640 1 -- 172.21.15.132:0/56624009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334105ee0 msgr2=0x7f83341985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:47.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.105+0000 7f833bea3640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334105ee0 0x7f83341985c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f832c00cad0 tx=0x7f832c00cf90 comp rx=0 tx=0).stop 2024-04-23T18:46:47.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.105+0000 7f833bea3640 1 -- 172.21.15.132:0/56624009 shutdown_connections 2024-04-23T18:46:47.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.105+0000 7f833bea3640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f83200a3d20 0x7f83200a61e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:47.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.105+0000 7f833bea3640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334105ee0 0x7f83341985c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:47.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.105+0000 7f833bea3640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334101b60 0x7f833419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:47.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.105+0000 7f833bea3640 1 --2- 172.21.15.132:0/56624009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83341011a0 0x7f833419e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:47.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.105+0000 7f833bea3640 1 -- 172.21.15.132:0/56624009 >> 172.21.15.132:0/56624009 conn(0x7f8334076c10 msgr2=0x7f833406a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:47.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.105+0000 7f833bea3640 1 -- 172.21.15.132:0/56624009 shutdown_connections 2024-04-23T18:46:47.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.105+0000 7f833bea3640 1 -- 172.21.15.132:0/56624009 wait complete. 2024-04-23T18:46:47.113 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T18:46:47.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.226+0000 7ff82ec7c640 1 -- 172.21.15.132:0/3351074524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff828100fc0 msgr2=0x7ff8281013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:47.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.226+0000 7ff82ec7c640 1 --2- 172.21.15.132:0/3351074524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff828100fc0 0x7ff8281013a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009900 tx=0x7ff81c02a510 comp rx=0 tx=0).stop 2024-04-23T18:46:47.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.226+0000 7ff82ec7c640 1 -- 172.21.15.132:0/3351074524 shutdown_connections 2024-04-23T18:46:47.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.226+0000 7ff82ec7c640 1 --2- 172.21.15.132:0/3351074524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8281018e0 0x7ff828105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:47.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.226+0000 7ff82ec7c640 1 --2- 172.21.15.132:0/3351074524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8281062e0 0x7ff828106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:47.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.226+0000 7ff82ec7c640 1 --2- 172.21.15.132:0/3351074524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff828100fc0 0x7ff8281013a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:47.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.226+0000 7ff82ec7c640 1 -- 172.21.15.132:0/3351074524 >> 172.21.15.132:0/3351074524 conn(0x7ff828077f50 msgr2=0x7ff828078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:47.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.226+0000 7ff82ec7c640 1 -- 172.21.15.132:0/3351074524 shutdown_connections 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.226+0000 7ff82ec7c640 1 -- 172.21.15.132:0/3351074524 wait complete. 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.226+0000 7ff82ec7c640 1 Processor -- start 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff82ec7c640 1 -- start start 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff82ec7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff828100fc0 0x7ff82819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff82ec7c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8281018e0 0x7ff82819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff82ec7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8281062e0 0x7ff8281a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff82ec7c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff8281187b0 con 0x7ff8281018e0 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff82ec7c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff828118630 con 0x7ff828100fc0 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff82ec7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff828118930 con 0x7ff8281062e0 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff827fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8281018e0 0x7ff82819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff827fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8281018e0 0x7ff82819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37954/0 (socket says 172.21.15.132:37954) 2024-04-23T18:46:47.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff827fff640 1 -- 172.21.15.132:0/2447640965 learned_addr learned my addr 172.21.15.132:0/2447640965 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:47.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff82ca18640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff828100fc0 0x7ff82819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:47.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff82d219640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8281062e0 0x7ff8281a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:47.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff827fff640 1 -- 172.21.15.132:0/2447640965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8281062e0 msgr2=0x7ff8281a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:47.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff827fff640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8281062e0 0x7ff8281a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:47.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff827fff640 1 -- 172.21.15.132:0/2447640965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff828100fc0 msgr2=0x7ff82819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:47.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff827fff640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff828100fc0 0x7ff82819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:47.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff827fff640 1 -- 172.21.15.132:0/2447640965 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8281a3dc0 con 0x7ff8281018e0 2024-04-23T18:46:47.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff827fff640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8281018e0 0x7ff82819f310 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff81400e9f0 tx=0x7ff81400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:47.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff825ffb640 1 -- 172.21.15.132:0/2447640965 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81400cd80 con 0x7ff8281018e0 2024-04-23T18:46:47.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff825ffb640 1 -- 172.21.15.132:0/2447640965 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8140054d0 con 0x7ff8281018e0 2024-04-23T18:46:47.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff82ec7c640 1 -- 172.21.15.132:0/2447640965 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8281a40b0 con 0x7ff8281018e0 2024-04-23T18:46:47.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.227+0000 7ff825ffb640 1 -- 172.21.15.132:0/2447640965 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814010690 con 0x7ff8281018e0 2024-04-23T18:46:47.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.228+0000 7ff82ec7c640 1 -- 172.21.15.132:0/2447640965 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8281ab940 con 0x7ff8281018e0 2024-04-23T18:46:47.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.229+0000 7ff82ec7c640 1 -- 172.21.15.132:0/2447640965 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff828068d70 con 0x7ff8281018e0 2024-04-23T18:46:47.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.230+0000 7ff825ffb640 1 -- 172.21.15.132:0/2447640965 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7ff814004e90 con 0x7ff8281018e0 2024-04-23T18:46:47.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.230+0000 7ff825ffb640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff8180a3d20 0x7ff8180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:47.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.230+0000 7ff825ffb640 1 -- 172.21.15.132:0/2447640965 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8965+0+0 (secure 0 0 0) 0x7ff8140d0db0 con 0x7ff8281018e0 2024-04-23T18:46:47.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.230+0000 7ff82ca18640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff8180a3d20 0x7ff8180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:47.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.231+0000 7ff82ca18640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff8180a3d20 0x7ff8180a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff81c031440 tx=0x7ff81c005b50 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:47.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.235+0000 7ff825ffb640 1 -- 172.21.15.132:0/2447640965 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff81409d290 con 0x7ff8281018e0 2024-04-23T18:46:47.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a[29174]: 2024-04-23T18:46:47.100+0000 7fbc0a641700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T18:46:47.361 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-g[70379]: ignoring --setuser ceph since I am not root 2024-04-23T18:46:47.361 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-g[70379]: ignoring --setgroup ceph since I am not root 2024-04-23T18:46:47.361 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-mds[70383]: -- 172.21.15.132:0/3626379963 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb64e2fa40 con 0x55cb64e02c00 2024-04-23T18:46:47.361 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-g[70379]: starting mds.g at 2024-04-23T18:46:47.361 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-a[70735]: ignoring --setuser ceph since I am not root 2024-04-23T18:46:47.361 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-a[70735]: ignoring --setgroup ceph since I am not root 2024-04-23T18:46:47.361 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-mds[70740]: -- 172.21.15.132:0/882832579 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd6b12ba40 con 0x55fd6b0fec00 2024-04-23T18:46:47.361 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-a[70735]: starting mds.a at 2024-04-23T18:46:47.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:47.452+0000 7ff82ec7c640 1 -- 172.21.15.132:0/2447640965 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff8281abbd0 con 0x7ff8281018e0 2024-04-23T18:46:47.449 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f[62879]: ignoring --setuser ceph since I am not root 2024-04-23T18:46:47.449 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f[62879]: ignoring --setgroup ceph since I am not root 2024-04-23T18:46:47.449 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mds[62883]: -- 172.21.15.157:0/2424357643 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5607eec7a4e0 con 0x5607eec50800 2024-04-23T18:46:47.449 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mds[62883]: -- 172.21.15.157:0/2424357643 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5607eec7da40 con 0x5607eec50800 2024-04-23T18:46:47.449 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f[62879]: starting mds.f at 2024-04-23T18:46:47.449 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l[62558]: ignoring --setuser ceph since I am not root 2024-04-23T18:46:47.449 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l[62558]: ignoring --setgroup ceph since I am not root 2024-04-23T18:46:47.449 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mds[62562]: -- 172.21.15.157:0/955576004 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559288537a40 con 0x55928850a800 2024-04-23T18:46:47.449 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l[62558]: starting mds.l at 2024-04-23T18:46:47.450 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-i[63503]: ignoring --setuser ceph since I am not root 2024-04-23T18:46:47.450 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-i[63503]: ignoring --setgroup ceph since I am not root 2024-04-23T18:46:47.450 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mds[63507]: -- 172.21.15.157:0/1348148448 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9f69fda40 con 0x55f9f69d1000 2024-04-23T18:46:47.450 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-i[63503]: starting mds.i at 2024-04-23T18:46:47.450 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-c[63192]: ignoring --setuser ceph since I am not root 2024-04-23T18:46:47.450 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-c[63192]: ignoring --setgroup ceph since I am not root 2024-04-23T18:46:47.450 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mds[63196]: -- 172.21.15.157:0/2461874931 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ef8341c4e0 con 0x55ef833f2800 2024-04-23T18:46:47.451 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mds[63196]: -- 172.21.15.157:0/2461874931 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef8341fa40 con 0x55ef833f2800 2024-04-23T18:46:47.451 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-c[63192]: starting mds.c at 2024-04-23T18:46:47.513 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-h[62217]: ignoring --setuser ceph since I am not root 2024-04-23T18:46:47.514 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-h[62217]: ignoring --setgroup ceph since I am not root 2024-04-23T18:46:47.514 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-mds[62221]: -- 172.21.15.148:0/3476272459 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562212405a40 con 0x5622123d8c00 2024-04-23T18:46:47.514 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-h[62217]: starting mds.h at 2024-04-23T18:46:47.514 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-k[61588]: ignoring --setuser ceph since I am not root 2024-04-23T18:46:47.515 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-k[61588]: ignoring --setgroup ceph since I am not root 2024-04-23T18:46:47.515 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-k[61588]: starting mds.k at 2024-04-23T18:46:47.515 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b[62525]: ignoring --setuser ceph since I am not root 2024-04-23T18:46:47.515 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b[62525]: ignoring --setgroup ceph since I am not root 2024-04-23T18:46:47.515 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-mds[62529]: -- 172.21.15.148:0/852586481 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e49dc9a40 con 0x564e49d9d000 2024-04-23T18:46:47.515 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b[62525]: starting mds.b at 2024-04-23T18:46:47.515 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-e[61906]: ignoring --setuser ceph since I am not root 2024-04-23T18:46:47.516 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-e[61906]: ignoring --setgroup ceph since I am not root 2024-04-23T18:46:47.516 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-mds[61910]: -- 172.21.15.148:0/2645556597 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636eca4fa40 con 0x5636eca22800 2024-04-23T18:46:47.516 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-e[61906]: starting mds.e at 2024-04-23T18:46:48.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/756449642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:46:48.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/56624009' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T18:46:48.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-mon[32311]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T18:46:48.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-mon[32311]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T18:46:48.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/56624009' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T18:46:48.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-mon[32311]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T18:46:48.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-mon[32311]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-23T18:46:48.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:47 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2447640965' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:46:48.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/756449642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:46:48.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/56624009' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T18:46:48.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mon[32580]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T18:46:48.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mon[32580]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T18:46:48.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/56624009' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T18:46:48.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mon[32580]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T18:46:48.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mon[32580]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-23T18:46:48.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:47 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2447640965' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:46:48.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.103+0000 7ff825ffb640 1 -- 172.21.15.132:0/2447640965 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7ff81409cb90 con 0x7ff8281018e0 2024-04-23T18:46:48.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.106+0000 7ff82ec7c640 1 -- 172.21.15.132:0/2447640965 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff8180a3d20 msgr2=0x7ff8180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:48.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.106+0000 7ff82ec7c640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff8180a3d20 0x7ff8180a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff81c031440 tx=0x7ff81c005b50 comp rx=0 tx=0).stop 2024-04-23T18:46:48.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.106+0000 7ff82ec7c640 1 -- 172.21.15.132:0/2447640965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8281018e0 msgr2=0x7ff82819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:48.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.106+0000 7ff82ec7c640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8281018e0 0x7ff82819f310 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff81400e9f0 tx=0x7ff81400edf0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.106+0000 7ff82ec7c640 1 -- 172.21.15.132:0/2447640965 shutdown_connections 2024-04-23T18:46:48.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.106+0000 7ff82ec7c640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7ff8180a3d20 0x7ff8180a61e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.106+0000 7ff82ec7c640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8281062e0 0x7ff8281a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.106+0000 7ff82ec7c640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8281018e0 0x7ff82819f310 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.106+0000 7ff82ec7c640 1 --2- 172.21.15.132:0/2447640965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff828100fc0 0x7ff82819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.106+0000 7ff82ec7c640 1 -- 172.21.15.132:0/2447640965 >> 172.21.15.132:0/2447640965 conn(0x7ff828077f50 msgr2=0x7ff828103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:48.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.106+0000 7ff82ec7c640 1 -- 172.21.15.132:0/2447640965 shutdown_connections 2024-04-23T18:46:48.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.106+0000 7ff82ec7c640 1 -- 172.21.15.132:0/2447640965 wait complete. 2024-04-23T18:46:48.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/756449642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:46:48.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/56624009' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T18:46:48.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-mon[29178]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T18:46:48.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-mon[29178]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T18:46:48.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/56624009' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T18:46:48.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-mon[29178]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T18:46:48.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-mon[29178]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-23T18:46:48.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:47 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2447640965' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:46:48.113 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T18:46:48.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.226+0000 7fcb21715640 1 -- 172.21.15.132:0/3023516224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb1c106160 msgr2=0x7fcb1c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:48.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.226+0000 7fcb21715640 1 --2- 172.21.15.132:0/3023516224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb1c106160 0x7fcb1c10dd30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009ca0 tx=0x7fcb1402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.226+0000 7fcb21715640 1 -- 172.21.15.132:0/3023516224 shutdown_connections 2024-04-23T18:46:48.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.226+0000 7fcb21715640 1 --2- 172.21.15.132:0/3023516224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb1c101a40 0x7fcb1c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.226+0000 7fcb21715640 1 --2- 172.21.15.132:0/3023516224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb1c106160 0x7fcb1c10dd30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.226+0000 7fcb21715640 1 --2- 172.21.15.132:0/3023516224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c101090 0x7fcb1c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.226+0000 7fcb21715640 1 -- 172.21.15.132:0/3023516224 >> 172.21.15.132:0/3023516224 conn(0x7fcb1c077f50 msgr2=0x7fcb1c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:48.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.226+0000 7fcb21715640 1 -- 172.21.15.132:0/3023516224 shutdown_connections 2024-04-23T18:46:48.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.226+0000 7fcb21715640 1 -- 172.21.15.132:0/3023516224 wait complete. 2024-04-23T18:46:48.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb21715640 1 Processor -- start 2024-04-23T18:46:48.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb21715640 1 -- start start 2024-04-23T18:46:48.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb21715640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb1c101090 0x7fcb1c1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:48.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb21715640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb1c106160 0x7fcb1c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:48.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb21715640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c19be90 0x7fcb1c19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:48.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb21715640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c1110d0 con 0x7fcb1c106160 2024-04-23T18:46:48.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb21715640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c110f50 con 0x7fcb1c19be90 2024-04-23T18:46:48.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb21715640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c111250 con 0x7fcb1c101090 2024-04-23T18:46:48.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb13fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb1c106160 0x7fcb1c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:48.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb13fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb1c106160 0x7fcb1c197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37968/0 (socket says 172.21.15.132:37968) 2024-04-23T18:46:48.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb13fff640 1 -- 172.21.15.132:0/2809323077 learned_addr learned my addr 172.21.15.132:0/2809323077 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:48.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb1b7fe640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c19be90 0x7fcb1c19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:48.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb1affd640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb1c101090 0x7fcb1c1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:48.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb13fff640 1 -- 172.21.15.132:0/2809323077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb1c101090 msgr2=0x7fcb1c1975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:48.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb13fff640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb1c101090 0x7fcb1c1975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb13fff640 1 -- 172.21.15.132:0/2809323077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c19be90 msgr2=0x7fcb1c19c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:48.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb13fff640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c19be90 0x7fcb1c19c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb13fff640 1 -- 172.21.15.132:0/2809323077 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c19c980 con 0x7fcb1c106160 2024-04-23T18:46:48.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb1b7fe640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c19be90 0x7fcb1c19c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:48.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb1affd640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb1c101090 0x7fcb1c1975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:46:48.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.227+0000 7fcb13fff640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb1c106160 0x7fcb1c197b10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400c970 tx=0x7fcb0400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:48.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.228+0000 7fcb18ff9640 1 -- 172.21.15.132:0/2809323077 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04007bb0 con 0x7fcb1c106160 2024-04-23T18:46:48.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.228+0000 7fcb18ff9640 1 -- 172.21.15.132:0/2809323077 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb04007d50 con 0x7fcb1c106160 2024-04-23T18:46:48.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.228+0000 7fcb21715640 1 -- 172.21.15.132:0/2809323077 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1c19cc70 con 0x7fcb1c106160 2024-04-23T18:46:48.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.228+0000 7fcb18ff9640 1 -- 172.21.15.132:0/2809323077 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04018730 con 0x7fcb1c106160 2024-04-23T18:46:48.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.228+0000 7fcb21715640 1 -- 172.21.15.132:0/2809323077 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1c1a4410 con 0x7fcb1c106160 2024-04-23T18:46:48.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.229+0000 7fcb21715640 1 -- 172.21.15.132:0/2809323077 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1c068d70 con 0x7fcb1c106160 2024-04-23T18:46:48.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.230+0000 7fcb18ff9640 1 -- 172.21.15.132:0/2809323077 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fcb0400f990 con 0x7fcb1c106160 2024-04-23T18:46:48.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.230+0000 7fcb18ff9640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcb080a3d20 0x7fcb080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:48.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.230+0000 7fcb1affd640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcb080a3d20 0x7fcb080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:48.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.230+0000 7fcb18ff9640 1 -- 172.21.15.132:0/2809323077 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8965+0+0 (secure 0 0 0) 0x7fcb040d0ac0 con 0x7fcb1c106160 2024-04-23T18:46:48.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.231+0000 7fcb1affd640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcb080a3d20 0x7fcb080a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00baa0 tx=0x7fcb0c005d90 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:48.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.235+0000 7fcb18ff9640 1 -- 172.21.15.132:0/2809323077 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb04014030 con 0x7fcb1c106160 2024-04-23T18:46:48.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:48.413+0000 7fcb21715640 1 -- 172.21.15.132:0/2809323077 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcb1c101a40 con 0x7fcb1c106160 2024-04-23T18:46:49.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: pgmap v333: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:49.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T18:46:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T18:46:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: Cluster is now healthy 2024-04-23T18:46:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2447640965' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:46:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.132:6838/3315632727,v1:172.21.15.132:6839/3315632727] up:boot 2024-04-23T18:46:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.157:6832/280185750,v1:172.21.15.157:6834/280185750] up:boot 2024-04-23T18:46:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.157:6833/3949865582,v1:172.21.15.157:6835/3949865582] up:boot 2024-04-23T18:46:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.132:6840/1788890453,v1:172.21.15.132:6841/1788890453] up:boot 2024-04-23T18:46:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.148:6832/1974674036,v1:172.21.15.148:6834/1974674036] up:boot 2024-04-23T18:46:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.148:6833/2851318713,v1:172.21.15.148:6835/2851318713] up:boot 2024-04-23T18:46:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.157:6836/1778565595,v1:172.21.15.157:6837/1778565595] up:boot 2024-04-23T18:46:49.014 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.157:6838/808595962,v1:172.21.15.157:6839/808595962] up:boot 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.148:6836/4054871382,v1:172.21.15.148:6838/4054871382] up:boot 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: mds.? [v2:172.21.15.148:6837/1904353078,v1:172.21.15.148:6839/1904353078] up:boot 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: fsmap 12 up:standby 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:46:49.015 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:46:49.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:46:49.016 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:48 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2809323077' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:46:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: pgmap v333: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T18:46:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T18:46:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: Cluster is now healthy 2024-04-23T18:46:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2447640965' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:46:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.132:6838/3315632727,v1:172.21.15.132:6839/3315632727] up:boot 2024-04-23T18:46:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.157:6832/280185750,v1:172.21.15.157:6834/280185750] up:boot 2024-04-23T18:46:49.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.157:6833/3949865582,v1:172.21.15.157:6835/3949865582] up:boot 2024-04-23T18:46:49.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.132:6840/1788890453,v1:172.21.15.132:6841/1788890453] up:boot 2024-04-23T18:46:49.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.148:6832/1974674036,v1:172.21.15.148:6834/1974674036] up:boot 2024-04-23T18:46:49.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.148:6833/2851318713,v1:172.21.15.148:6835/2851318713] up:boot 2024-04-23T18:46:49.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.157:6836/1778565595,v1:172.21.15.157:6837/1778565595] up:boot 2024-04-23T18:46:49.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.157:6838/808595962,v1:172.21.15.157:6839/808595962] up:boot 2024-04-23T18:46:49.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.148:6836/4054871382,v1:172.21.15.148:6838/4054871382] up:boot 2024-04-23T18:46:49.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: mds.? [v2:172.21.15.148:6837/1904353078,v1:172.21.15.148:6839/1904353078] up:boot 2024-04-23T18:46:49.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: fsmap 12 up:standby 2024-04-23T18:46:49.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:46:49.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:46:49.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:46:49.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:46:49.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:46:49.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:46:49.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:46:49.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:46:49.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:46:49.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:46:49.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:46:49.041 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:48 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2809323077' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:46:49.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.105+0000 7fcb18ff9640 1 -- 172.21.15.132:0/2809323077 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7fcb0409c950 con 0x7fcb1c106160 2024-04-23T18:46:49.098 INFO:teuthology.orchestra.run.smithi132.stderr:pool 'cephfs_metadata' removed 2024-04-23T18:46:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.109+0000 7fcb21715640 1 -- 172.21.15.132:0/2809323077 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcb080a3d20 msgr2=0x7fcb080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:49.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.109+0000 7fcb21715640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcb080a3d20 0x7fcb080a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00baa0 tx=0x7fcb0c005d90 comp rx=0 tx=0).stop 2024-04-23T18:46:49.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.109+0000 7fcb21715640 1 -- 172.21.15.132:0/2809323077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb1c106160 msgr2=0x7fcb1c197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:49.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.109+0000 7fcb21715640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb1c106160 0x7fcb1c197b10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400c970 tx=0x7fcb0400ce30 comp rx=0 tx=0).stop 2024-04-23T18:46:49.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.109+0000 7fcb21715640 1 -- 172.21.15.132:0/2809323077 shutdown_connections 2024-04-23T18:46:49.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.109+0000 7fcb21715640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7fcb080a3d20 0x7fcb080a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:49.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.109+0000 7fcb21715640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c19be90 0x7fcb1c19c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:49.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.109+0000 7fcb21715640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb1c106160 0x7fcb1c197b10 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:49.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.109+0000 7fcb21715640 1 --2- 172.21.15.132:0/2809323077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb1c101090 0x7fcb1c1975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:49.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.109+0000 7fcb21715640 1 -- 172.21.15.132:0/2809323077 >> 172.21.15.132:0/2809323077 conn(0x7fcb1c077f50 msgr2=0x7fcb1c0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:49.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.110+0000 7fcb21715640 1 -- 172.21.15.132:0/2809323077 shutdown_connections 2024-04-23T18:46:49.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.110+0000 7fcb21715640 1 -- 172.21.15.132:0/2809323077 wait complete. 2024-04-23T18:46:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: pgmap v333: 97 pgs: 97 active+clean; 9.8 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-23T18:46:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T18:46:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T18:46:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: Cluster is now healthy 2024-04-23T18:46:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2447640965' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:46:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.132:6838/3315632727,v1:172.21.15.132:6839/3315632727] up:boot 2024-04-23T18:46:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.157:6832/280185750,v1:172.21.15.157:6834/280185750] up:boot 2024-04-23T18:46:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.157:6833/3949865582,v1:172.21.15.157:6835/3949865582] up:boot 2024-04-23T18:46:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.132:6840/1788890453,v1:172.21.15.132:6841/1788890453] up:boot 2024-04-23T18:46:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.148:6832/1974674036,v1:172.21.15.148:6834/1974674036] up:boot 2024-04-23T18:46:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.148:6833/2851318713,v1:172.21.15.148:6835/2851318713] up:boot 2024-04-23T18:46:49.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.157:6836/1778565595,v1:172.21.15.157:6837/1778565595] up:boot 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.157:6838/808595962,v1:172.21.15.157:6839/808595962] up:boot 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.148:6836/4054871382,v1:172.21.15.148:6838/4054871382] up:boot 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: mds.? [v2:172.21.15.148:6837/1904353078,v1:172.21.15.148:6839/1904353078] up:boot 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: fsmap 12 up:standby 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:46:49.112 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:46:49.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='mgr.15366 172.21.15.132:0/3740275009' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:46:49.113 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:48 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2809323077' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:46:49.117 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T18:46:49.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.235+0000 7f86fd0e8640 1 -- 172.21.15.132:0/4148348676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86f8101b40 msgr2=0x7f86f810dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:49.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.235+0000 7f86fd0e8640 1 --2- 172.21.15.132:0/4148348676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86f8101b40 0x7f86f810dc80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009cd0 tx=0x7f86f002a8e0 comp rx=0 tx=0).stop 2024-04-23T18:46:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.235+0000 7f86fd0e8640 1 -- 172.21.15.132:0/4148348676 shutdown_connections 2024-04-23T18:46:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.235+0000 7f86fd0e8640 1 --2- 172.21.15.132:0/4148348676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86f8101b40 0x7f86f810dc80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.235+0000 7f86fd0e8640 1 --2- 172.21.15.132:0/4148348676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86f81011a0 0x7f86f8101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.235+0000 7f86fd0e8640 1 --2- 172.21.15.132:0/4148348676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f8107030 0x7f86f8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.235+0000 7f86fd0e8640 1 -- 172.21.15.132:0/4148348676 >> 172.21.15.132:0/4148348676 conn(0x7f86f8076c50 msgr2=0x7f86f8077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.235+0000 7f86fd0e8640 1 -- 172.21.15.132:0/4148348676 shutdown_connections 2024-04-23T18:46:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.235+0000 7f86fd0e8640 1 -- 172.21.15.132:0/4148348676 wait complete. 2024-04-23T18:46:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86fd0e8640 1 Processor -- start 2024-04-23T18:46:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86fd0e8640 1 -- start start 2024-04-23T18:46:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86fd0e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86f81011a0 0x7f86f8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86fd0e8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86f8101b40 0x7f86f8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86fd0e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f8107030 0x7f86f819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86fd0e8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81110c0 con 0x7f86f8101b40 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86fd0e8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8110f40 con 0x7f86f81011a0 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86fd0e8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8111240 con 0x7f86f8107030 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86f659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86f8101b40 0x7f86f8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86f659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86f8101b40 0x7f86f8198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37986/0 (socket says 172.21.15.132:37986) 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86f6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86f81011a0 0x7f86f8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86f659c640 1 -- 172.21.15.132:0/1228875346 learned_addr learned my addr 172.21.15.132:0/1228875346 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.236+0000 7f86f759e640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f8107030 0x7f86f819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.237+0000 7f86f659c640 1 -- 172.21.15.132:0/1228875346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f8107030 msgr2=0x7f86f819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.237+0000 7f86f659c640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f8107030 0x7f86f819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.237+0000 7f86f659c640 1 -- 172.21.15.132:0/1228875346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86f81011a0 msgr2=0x7f86f8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.237+0000 7f86f659c640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86f81011a0 0x7f86f8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.237+0000 7f86f659c640 1 -- 172.21.15.132:0/1228875346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f819d6d0 con 0x7f86f8101b40 2024-04-23T18:46:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.237+0000 7f86f659c640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86f8101b40 0x7f86f8198c70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00c890 tx=0x7f86ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:49.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.237+0000 7f86dffff640 1 -- 172.21.15.132:0/1228875346 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec004220 con 0x7f86f8101b40 2024-04-23T18:46:49.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.237+0000 7f86dffff640 1 -- 172.21.15.132:0/1228875346 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec0043c0 con 0x7f86f8101b40 2024-04-23T18:46:49.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.237+0000 7f86fd0e8640 1 -- 172.21.15.132:0/1228875346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f819d9c0 con 0x7f86f8101b40 2024-04-23T18:46:49.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.237+0000 7f86dffff640 1 -- 172.21.15.132:0/1228875346 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec004a00 con 0x7f86f8101b40 2024-04-23T18:46:49.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.237+0000 7f86fd0e8640 1 -- 172.21.15.132:0/1228875346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f81a5250 con 0x7f86f8101b40 2024-04-23T18:46:49.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.238+0000 7f86fd0e8640 1 -- 172.21.15.132:0/1228875346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f8102590 con 0x7f86f8101b40 2024-04-23T18:46:49.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.239+0000 7f86dffff640 1 -- 172.21.15.132:0/1228875346 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f86ec0079b0 con 0x7f86f8101b40 2024-04-23T18:46:49.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.239+0000 7f86dffff640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f86e00a3d20 0x7f86e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:49.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.240+0000 7f86f6d9d640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f86e00a3d20 0x7f86e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:49.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.240+0000 7f86dffff640 1 -- 172.21.15.132:0/1228875346 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8411+0+0 (secure 0 0 0) 0x7f86ec0d04e0 con 0x7f86f8101b40 2024-04-23T18:46:49.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.240+0000 7f86f6d9d640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f86e00a3d20 0x7f86e00a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005d00 tx=0x7f86e800b410 comp rx=0 tx=0).ready entity=mgr.15366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:49.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.244+0000 7f86dffff640 1 -- 172.21.15.132:0/1228875346 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f86ec014030 con 0x7f86f8101b40 2024-04-23T18:46:49.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:49.431+0000 7f86fd0e8640 1 -- 172.21.15.132:0/1228875346 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f86f81a5510 con 0x7f86f8101b40 2024-04-23T18:46:50.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.108+0000 7f86dffff640 1 -- 172.21.15.132:0/1228875346 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7f86ec09cbf0 con 0x7f86f8101b40 2024-04-23T18:46:50.099 INFO:teuthology.orchestra.run.smithi132.stderr:pool 'cephfs_data' removed 2024-04-23T18:46:50.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.111+0000 7f86fd0e8640 1 -- 172.21.15.132:0/1228875346 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f86e00a3d20 msgr2=0x7f86e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:50.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.111+0000 7f86fd0e8640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f86e00a3d20 0x7f86e00a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005d00 tx=0x7f86e800b410 comp rx=0 tx=0).stop 2024-04-23T18:46:50.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.111+0000 7f86fd0e8640 1 -- 172.21.15.132:0/1228875346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86f8101b40 msgr2=0x7f86f8198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:50.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.111+0000 7f86fd0e8640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86f8101b40 0x7f86f8198c70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00c890 tx=0x7f86ec00cd50 comp rx=0 tx=0).stop 2024-04-23T18:46:50.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.112+0000 7f86fd0e8640 1 -- 172.21.15.132:0/1228875346 shutdown_connections 2024-04-23T18:46:50.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.112+0000 7f86fd0e8640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.132:6800/3428544997,v1:172.21.15.132:6801/3428544997] conn(0x7f86e00a3d20 0x7f86e00a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:50.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.112+0000 7f86fd0e8640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86f8107030 0x7f86f819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:50.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.112+0000 7f86fd0e8640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86f8101b40 0x7f86f8198c70 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:50.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.112+0000 7f86fd0e8640 1 --2- 172.21.15.132:0/1228875346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86f81011a0 0x7f86f8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:50.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.112+0000 7f86fd0e8640 1 -- 172.21.15.132:0/1228875346 >> 172.21.15.132:0/1228875346 conn(0x7f86f8076c50 msgr2=0x7f86f8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:50.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.112+0000 7f86fd0e8640 1 -- 172.21.15.132:0/1228875346 shutdown_connections 2024-04-23T18:46:50.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-23T18:46:50.112+0000 7f86fd0e8640 1 -- 172.21.15.132:0/1228875346 wait complete. 2024-04-23T18:46:50.118 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:46:50.143 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:46:50.175 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:46:50.205 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-23T18:46:50.205 DEBUG:teuthology.orchestra.run.smithi132:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:46:50.220 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:46:50.236 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:46:50.260 INFO:tasks.cephadm:Stopping all daemons... 2024-04-23T18:46:50.261 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-23T18:46:50.261 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.a 2024-04-23T18:46:50.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:50 smithi132 ceph-mon[29178]: pgmap v335: 97 pgs: 97 active+clean; 9.8 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-23T18:46:50.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:50 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/2809323077' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T18:46:50.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:50 smithi132 ceph-mon[29178]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T18:46:50.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:50 smithi132 ceph-mon[29178]: from='client.? 172.21.15.132:0/1228875346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:46:50.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:50 smithi132 systemd[1]: Stopping Ceph mon.a for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:46:50.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:50 smithi148 ceph-mon[32311]: pgmap v335: 97 pgs: 97 active+clean; 9.8 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-23T18:46:50.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:50 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/2809323077' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T18:46:50.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:50 smithi148 ceph-mon[32311]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T18:46:50.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:50 smithi148 ceph-mon[32311]: from='client.? 172.21.15.132:0/1228875346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:46:50.538 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:50 smithi157 ceph-mon[32580]: pgmap v335: 97 pgs: 97 active+clean; 9.8 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-23T18:46:50.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:50 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/2809323077' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T18:46:50.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:50 smithi157 ceph-mon[32580]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T18:46:50.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:50 smithi157 ceph-mon[32580]: from='client.? 172.21.15.132:0/1228875346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:46:50.827 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:50 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a[29174]: 2024-04-23T18:46:50.569+0000 7fbc0fe4c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:46:50.827 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:50 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a[29174]: 2024-04-23T18:46:50.569+0000 7fbc0fe4c700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-23T18:46:51.110 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:50 smithi132 podman[89065]: 2024-04-23 18:46:50.835805945 +0000 UTC m=+0.378781988 container died 2bb1a8944af28005f4862de28709bdc377209a0bf3acbc624102a9c218734d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-23T18:46:51.488 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:51 smithi132 podman[89065]: 2024-04-23 18:46:51.237655235 +0000 UTC m=+0.780631280 container cleanup 2bb1a8944af28005f4862de28709bdc377209a0bf3acbc624102a9c218734d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416) 2024-04-23T18:46:51.488 INFO:journalctl@ceph.mon.a.smithi132.stdout:Apr 23 18:46:51 smithi132 bash[89065]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-a 2024-04-23T18:46:51.617 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.a.service' 2024-04-23T18:46:51.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:46:51.662 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-23T18:46:51.662 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-23T18:46:51.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.b 2024-04-23T18:46:52.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:51 smithi148 systemd[1]: Stopping Ceph mon.b for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:46:52.471 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:52 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-b[32307]: 2024-04-23T18:46:52.213+0000 7f52ee159700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:46:52.471 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:52 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-b[32307]: 2024-04-23T18:46:52.213+0000 7f52ee159700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-23T18:46:52.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:52 smithi148 podman[67947]: 2024-04-23 18:46:52.480712411 +0000 UTC m=+0.615135985 container died dff4ea7f056dc6a5d336cf60aedc4e49ce43d3adfa4ba52080973ec6330a1817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-04-23T18:46:53.146 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:52 smithi148 podman[67947]: 2024-04-23 18:46:52.895889099 +0000 UTC m=+1.030312668 container cleanup dff4ea7f056dc6a5d336cf60aedc4e49ce43d3adfa4ba52080973ec6330a1817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-b, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-23T18:46:53.146 INFO:journalctl@ceph.mon.b.smithi148.stdout:Apr 23 18:46:52 smithi148 bash[67947]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-b 2024-04-23T18:46:53.362 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.b.service' 2024-04-23T18:46:53.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:46:53.412 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-23T18:46:53.412 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-23T18:46:53.412 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.c 2024-04-23T18:46:53.724 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:53 smithi157 systemd[1]: Stopping Ceph mon.c for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:46:54.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:53 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-c[32564]: 2024-04-23T18:46:53.732+0000 7ffa97748700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:46:54.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:53 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-c[32564]: 2024-04-23T18:46:53.732+0000 7ffa97748700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-23T18:46:54.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:53 smithi157 podman[68977]: 2024-04-23 18:46:53.876592434 +0000 UTC m=+0.263998655 container died ff4c6dfe43a861932405403ca679b9ce7acadf0fe77d2be10a7e5d07eb678b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-23T18:46:54.510 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:54 smithi157 podman[68977]: 2024-04-23 18:46:54.28491376 +0000 UTC m=+0.672319982 container cleanup ff4c6dfe43a861932405403ca679b9ce7acadf0fe77d2be10a7e5d07eb678b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-23T18:46:54.510 INFO:journalctl@ceph.mon.c.smithi157.stdout:Apr 23 18:46:54 smithi157 bash[68977]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mon-c 2024-04-23T18:46:54.648 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mon.c.service' 2024-04-23T18:46:54.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:46:54.716 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-23T18:46:54.716 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-23T18:46:54.716 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.x 2024-04-23T18:46:55.041 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:46:54 smithi132 systemd[1]: Stopping Ceph mgr.x for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:46:55.360 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:46:55 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:46:55.049+0000 7fa7ab441700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:46:55.360 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:46:55 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x[29412]: 2024-04-23T18:46:55.049+0000 7fa7ab441700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-23T18:46:55.360 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:46:55 smithi132 podman[89188]: 2024-04-23 18:46:55.081328881 +0000 UTC m=+0.158251586 container died 390d7bfdf89bc8ba87bcf5b271bf04255ee6b1ae71bf854b5e1e06d58485be30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:46:55.693 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:46:55 smithi132 podman[89188]: 2024-04-23 18:46:55.476507726 +0000 UTC m=+0.553430430 container cleanup 390d7bfdf89bc8ba87bcf5b271bf04255ee6b1ae71bf854b5e1e06d58485be30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-23T18:46:55.694 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:46:55 smithi132 bash[89188]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x 2024-04-23T18:46:55.934 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-23T18:46:55.955 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:46:55 smithi132 podman[89203]: 2024-04-23 18:46:55.703574119 +0000 UTC m=+0.620487273 container remove 390d7bfdf89bc8ba87bcf5b271bf04255ee6b1ae71bf854b5e1e06d58485be30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-23T18:46:55.955 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:46:55 smithi132 systemd[1]: ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.x.service: Deactivated successfully. 2024-04-23T18:46:55.955 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:46:55 smithi132 systemd[1]: Stopped Ceph mgr.x for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:46:55.955 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Apr 23 18:46:55 smithi132 systemd[1]: ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.x.service: Consumed 1min 22.206s CPU time. 2024-04-23T18:46:55.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:46:55.968 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-23T18:46:55.968 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-23T18:46:55.969 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.y 2024-04-23T18:46:56.263 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:46:56 smithi148 systemd[1]: Stopping Ceph mgr.y for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:46:56.567 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:46:56 smithi148 podman[68067]: 2024-04-23 18:46:56.287982013 +0000 UTC m=+0.114247713 container died 203519ea92c8998d25c6af184c8ab0479f15d90e50373982d0f134e0d9054e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-23T18:46:56.926 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:46:56 smithi148 podman[68067]: 2024-04-23 18:46:56.676342262 +0000 UTC m=+0.502607969 container cleanup 203519ea92c8998d25c6af184c8ab0479f15d90e50373982d0f134e0d9054e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:46:56.926 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Apr 23 18:46:56 smithi148 bash[68067]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-y 2024-04-23T18:46:57.089 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-23T18:46:57.122 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:46:57.122 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-23T18:46:57.123 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-23T18:46:57.123 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.z 2024-04-23T18:46:57.475 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:46:57 smithi157 systemd[1]: Stopping Ceph mgr.z for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:46:57.781 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:46:57 smithi157 podman[69100]: 2024-04-23 18:46:57.485377923 +0000 UTC m=+0.153120893 container died 7f74d05ed853e73da54e37f5a908a6a74f1810cb3047d1d2c11842f5c474b9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-23T18:46:58.038 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:46:57 smithi157 podman[69100]: 2024-04-23 18:46:57.907764813 +0000 UTC m=+0.575507787 container cleanup 7f74d05ed853e73da54e37f5a908a6a74f1810cb3047d1d2c11842f5c474b9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-23T18:46:58.039 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Apr 23 18:46:57 smithi157 bash[69100]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mgr-z 2024-04-23T18:46:58.360 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-23T18:46:58.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:46:58.393 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-23T18:46:58.394 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-23T18:46:58.394 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.0 2024-04-23T18:46:58.697 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:46:58 smithi132 systemd[1]: Stopping Ceph osd.0 for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:46:58.958 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:46:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0[43945]: 2024-04-23T18:46:58.706+0000 7f2bce0a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:46:58.959 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:46:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0[43945]: 2024-04-23T18:46:58.706+0000 7f2bce0a4700 -1 osd.0 102 *** Got signal Terminated *** 2024-04-23T18:46:58.959 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:46:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0[43945]: 2024-04-23T18:46:58.706+0000 7f2bce0a4700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:47:04.493 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:47:04 smithi132 podman[89312]: 2024-04-23 18:47:04.191518474 +0000 UTC m=+5.602364470 container died 7226ebbf5a58517f509906bc2f89a64a41c121c49b02b29c9111a1a47dc42700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-23T18:47:04.860 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:47:04 smithi132 podman[89312]: 2024-04-23 18:47:04.5993809 +0000 UTC m=+6.010226926 container cleanup 7226ebbf5a58517f509906bc2f89a64a41c121c49b02b29c9111a1a47dc42700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:47:04.861 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:47:04 smithi132 bash[89312]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0 2024-04-23T18:47:05.123 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:47:04 smithi132 podman[89341]: 2024-04-23 18:47:04.875927407 +0000 UTC m=+0.679809034 container remove 7226ebbf5a58517f509906bc2f89a64a41c121c49b02b29c9111a1a47dc42700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-23T18:47:05.389 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:47:05 smithi132 podman[89429]: 2024-04-23 18:47:05.128852736 +0000 UTC m=+0.019243972 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:47:05.389 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:47:05 smithi132 podman[89429]: 2024-04-23 18:47:05.260663956 +0000 UTC m=+0.151055223 container create b60b52d28a395fa4a263fffd89937e3b741b262e176a04869d4ae8ece7eda4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-23T18:47:05.785 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:47:05 smithi132 podman[89429]: 2024-04-23 18:47:05.496856966 +0000 UTC m=+0.387248189 container init b60b52d28a395fa4a263fffd89937e3b741b262e176a04869d4ae8ece7eda4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-23T18:47:05.785 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:47:05 smithi132 podman[89429]: 2024-04-23 18:47:05.500035568 +0000 UTC m=+0.390426795 container start b60b52d28a395fa4a263fffd89937e3b741b262e176a04869d4ae8ece7eda4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:47:05.786 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:47:05 smithi132 podman[89429]: 2024-04-23 18:47:05.571795195 +0000 UTC m=+0.462186434 container attach b60b52d28a395fa4a263fffd89937e3b741b262e176a04869d4ae8ece7eda4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-23T18:47:06.110 INFO:journalctl@ceph.osd.0.smithi132.stdout:Apr 23 18:47:05 smithi132 podman[89429]: 2024-04-23 18:47:05.795306543 +0000 UTC m=+0.685697768 container died b60b52d28a395fa4a263fffd89937e3b741b262e176a04869d4ae8ece7eda4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-23T18:47:06.457 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.0.service' 2024-04-23T18:47:06.496 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:47:06.496 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-23T18:47:06.496 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-23T18:47:06.496 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.1 2024-04-23T18:47:06.812 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:06 smithi148 systemd[1]: Stopping Ceph osd.1 for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:47:07.263 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:06 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1[36913]: 2024-04-23T18:47:06.820+0000 7ffa7bcf7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:47:07.263 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:06 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1[36913]: 2024-04-23T18:47:06.820+0000 7ffa7bcf7700 -1 osd.1 102 *** Got signal Terminated *** 2024-04-23T18:47:07.263 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:06 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1[36913]: 2024-04-23T18:47:06.820+0000 7ffa7bcf7700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:47:12.597 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:12 smithi148 podman[68188]: 2024-04-23 18:47:12.302179981 +0000 UTC m=+5.600314636 container died 6d28072d31d3b4e1d9ee8638e86f4de471e13846211651bcbb9fe6f4e1892e8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-23T18:47:12.964 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:12 smithi148 podman[68188]: 2024-04-23 18:47:12.697018836 +0000 UTC m=+5.995153498 container cleanup 6d28072d31d3b4e1d9ee8638e86f4de471e13846211651bcbb9fe6f4e1892e8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:47:12.964 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:12 smithi148 bash[68188]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1 2024-04-23T18:47:13.263 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:12 smithi148 podman[68205]: 2024-04-23 18:47:12.973553427 +0000 UTC m=+0.668810808 container remove 6d28072d31d3b4e1d9ee8638e86f4de471e13846211651bcbb9fe6f4e1892e8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-23T18:47:13.526 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:13 smithi148 podman[68299]: 2024-04-23 18:47:13.180199373 +0000 UTC m=+0.017739667 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:47:13.526 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:13 smithi148 podman[68299]: 2024-04-23 18:47:13.31606729 +0000 UTC m=+0.153607568 container create e24fea6a18a25fe5c081dc854fdcae7d91b583fce0e4844f40f324a407bfb971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-23T18:47:13.825 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:13 smithi148 podman[68299]: 2024-04-23 18:47:13.536065515 +0000 UTC m=+0.373605795 container init e24fea6a18a25fe5c081dc854fdcae7d91b583fce0e4844f40f324a407bfb971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:47:13.825 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:13 smithi148 podman[68299]: 2024-04-23 18:47:13.539703959 +0000 UTC m=+0.377244236 container start e24fea6a18a25fe5c081dc854fdcae7d91b583fce0e4844f40f324a407bfb971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-23T18:47:13.825 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:13 smithi148 podman[68299]: 2024-04-23 18:47:13.660609382 +0000 UTC m=+0.498149663 container attach e24fea6a18a25fe5c081dc854fdcae7d91b583fce0e4844f40f324a407bfb971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-04-23T18:47:14.196 INFO:journalctl@ceph.osd.1.smithi148.stdout:Apr 23 18:47:13 smithi148 podman[68299]: 2024-04-23 18:47:13.834849241 +0000 UTC m=+0.672389509 container died e24fea6a18a25fe5c081dc854fdcae7d91b583fce0e4844f40f324a407bfb971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-23T18:47:14.554 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.1.service' 2024-04-23T18:47:14.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:47:14.595 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-23T18:47:14.595 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-23T18:47:14.595 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.2 2024-04-23T18:47:14.911 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:14 smithi157 systemd[1]: Stopping Ceph osd.2 for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:47:15.288 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:14 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2[37705]: 2024-04-23T18:47:14.920+0000 7f4fec3fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:47:15.288 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:14 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2[37705]: 2024-04-23T18:47:14.920+0000 7f4fec3fb700 -1 osd.2 102 *** Got signal Terminated *** 2024-04-23T18:47:15.289 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:14 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2[37705]: 2024-04-23T18:47:14.920+0000 7f4fec3fb700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:47:20.539 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:20 smithi157 podman[69224]: 2024-04-23 18:47:20.284751322 +0000 UTC m=+5.486280763 container died 1eb04c4bad5605024a5a4127d6810d4ad0c0942d2e893b57e44e65e1e37947fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-23T18:47:20.938 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:20 smithi157 podman[69224]: 2024-04-23 18:47:20.662881497 +0000 UTC m=+5.864410937 container cleanup 1eb04c4bad5605024a5a4127d6810d4ad0c0942d2e893b57e44e65e1e37947fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-23T18:47:20.938 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:20 smithi157 bash[69224]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2 2024-04-23T18:47:21.276 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:20 smithi157 podman[69253]: 2024-04-23 18:47:20.947790082 +0000 UTC m=+0.660444861 container remove 1eb04c4bad5605024a5a4127d6810d4ad0c0942d2e893b57e44e65e1e37947fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-23T18:47:21.539 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:21 smithi157 podman[69344]: 2024-04-23 18:47:21.18896009 +0000 UTC m=+0.018283221 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:47:21.539 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:21 smithi157 podman[69344]: 2024-04-23 18:47:21.315544567 +0000 UTC m=+0.144867687 container create 2f3046d9cabbf05d0e1d8e98344bdf0308aa7fedf0d5b3dbe3c3b24563ca88ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-04-23T18:47:21.922 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:21 smithi157 podman[69344]: 2024-04-23 18:47:21.635299297 +0000 UTC m=+0.464622412 container init 2f3046d9cabbf05d0e1d8e98344bdf0308aa7fedf0d5b3dbe3c3b24563ca88ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True) 2024-04-23T18:47:21.922 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:21 smithi157 podman[69344]: 2024-04-23 18:47:21.638390491 +0000 UTC m=+0.467713608 container start 2f3046d9cabbf05d0e1d8e98344bdf0308aa7fedf0d5b3dbe3c3b24563ca88ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-23T18:47:21.922 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:21 smithi157 podman[69344]: 2024-04-23 18:47:21.718744721 +0000 UTC m=+0.548067840 container attach 2f3046d9cabbf05d0e1d8e98344bdf0308aa7fedf0d5b3dbe3c3b24563ca88ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-23T18:47:22.289 INFO:journalctl@ceph.osd.2.smithi157.stdout:Apr 23 18:47:21 smithi157 podman[69344]: 2024-04-23 18:47:21.931748547 +0000 UTC m=+0.761071668 container died 2f3046d9cabbf05d0e1d8e98344bdf0308aa7fedf0d5b3dbe3c3b24563ca88ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-23T18:47:22.624 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.2.service' 2024-04-23T18:47:22.662 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:47:22.662 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-23T18:47:22.662 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-23T18:47:22.663 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.3 2024-04-23T18:47:22.977 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:22 smithi132 systemd[1]: Stopping Ceph osd.3 for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:47:23.360 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:22 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3[50631]: 2024-04-23T18:47:22.985+0000 7f9349f9f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:47:23.360 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:22 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3[50631]: 2024-04-23T18:47:22.985+0000 7f9349f9f700 -1 osd.3 102 *** Got signal Terminated *** 2024-04-23T18:47:23.360 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:22 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3[50631]: 2024-04-23T18:47:22.985+0000 7f9349f9f700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:47:24.513 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:24 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:24.265+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:25.513 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:25 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:25.306+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:25.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:25 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:25.428+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:25.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:25 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:25.248+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:25.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:25 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:25.253+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:26.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:26 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:26.396+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:26.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:26 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:26.202+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:26.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:26 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:26.302+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:26.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:26 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:26.347+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:27.090 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:26 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3[50631]: 2024-04-23T18:47:26.823+0000 7f9346593700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.375046+0000 front 2024-04-23T18:47:02.375002+0000 (oldest deadline 2024-04-23T18:47:25.875161+0000) 2024-04-23T18:47:27.360 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:27 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:27.347+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:27.360 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:27 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:27.099+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:27.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:27 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:27.407+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:27.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:27 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:27.241+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:27.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:27 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:27.350+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:28.106 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:27 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3[50631]: 2024-04-23T18:47:27.817+0000 7f9346593700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.375046+0000 front 2024-04-23T18:47:02.375002+0000 (oldest deadline 2024-04-23T18:47:25.875161+0000) 2024-04-23T18:47:28.360 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:28 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:28.114+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:28.360 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:28 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:28.300+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:28.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:28 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:28.410+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:28.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:28 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:28.257+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:28.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:28 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:28.362+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:28.770 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:28 smithi132 podman[89538]: 2024-04-23 18:47:28.475736935 +0000 UTC m=+5.611619375 container died 36ade69625794022f55b24e86757c76dff5c3db5f5c82c0c6ff7d409357da45a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T18:47:29.110 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:28 smithi132 podman[89538]: 2024-04-23 18:47:28.879358438 +0000 UTC m=+6.015240870 container cleanup 36ade69625794022f55b24e86757c76dff5c3db5f5c82c0c6ff7d409357da45a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-23T18:47:29.110 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:28 smithi132 bash[89538]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3 2024-04-23T18:47:29.404 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:29 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:29.139+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:29.404 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:29 smithi132 podman[89558]: 2024-04-23 18:47:29.147510852 +0000 UTC m=+0.666000949 container remove 36ade69625794022f55b24e86757c76dff5c3db5f5c82c0c6ff7d409357da45a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-23T18:47:29.405 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:29 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:29.326+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:29.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:29 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:29.388+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:29.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:29 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:29.260+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:29.655 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:29 smithi132 podman[89654]: 2024-04-23 18:47:29.408586717 +0000 UTC m=+0.021638995 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:47:29.655 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:29 smithi132 podman[89654]: 2024-04-23 18:47:29.523707038 +0000 UTC m=+0.136759289 container create 0c7dabbbdc4118b55f4c4cde0124dbb303c3c5c0c06cb5d935e10d4c0e6d212d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-04-23T18:47:29.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:29 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:29.353+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:30.062 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:29 smithi132 podman[89654]: 2024-04-23 18:47:29.768854442 +0000 UTC m=+0.381906703 container init 0c7dabbbdc4118b55f4c4cde0124dbb303c3c5c0c06cb5d935e10d4c0e6d212d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-23T18:47:30.062 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:29 smithi132 podman[89654]: 2024-04-23 18:47:29.772242781 +0000 UTC m=+0.385295033 container start 0c7dabbbdc4118b55f4c4cde0124dbb303c3c5c0c06cb5d935e10d4c0e6d212d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-04-23T18:47:30.063 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:29 smithi132 podman[89654]: 2024-04-23 18:47:29.868550396 +0000 UTC m=+0.481602662 container attach 0c7dabbbdc4118b55f4c4cde0124dbb303c3c5c0c06cb5d935e10d4c0e6d212d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-23T18:47:30.063 INFO:journalctl@ceph.osd.3.smithi132.stdout:Apr 23 18:47:30 smithi132 podman[89654]: 2024-04-23 18:47:30.071760954 +0000 UTC m=+0.684813216 container died 0c7dabbbdc4118b55f4c4cde0124dbb303c3c5c0c06cb5d935e10d4c0e6d212d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-23T18:47:30.360 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:30 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:30.187+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:30.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:30 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:30.402+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:30.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:30 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:30.233+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:30.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:30 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:30.369+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:30.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:30 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:30.391+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:30.791 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.3.service' 2024-04-23T18:47:30.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:47:30.835 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-23T18:47:30.835 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-23T18:47:30.835 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.4 2024-04-23T18:47:31.224 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:30 smithi148 systemd[1]: Stopping Ceph osd.4 for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:47:31.513 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:31 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:31.411+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:31.513 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:31 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-23T18:47:31.234+0000 7ffbad560700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:47:31.513 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:31 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-23T18:47:31.234+0000 7ffbad560700 -1 osd.4 102 *** Got signal Terminated *** 2024-04-23T18:47:31.513 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:31 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-23T18:47:31.234+0000 7ffbad560700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:47:31.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:31 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:31.387+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:31.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:31 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:31.251+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:31.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:31 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:31.224+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:31.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:31 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:31.355+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:32.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:32 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:32.373+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:32.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:32 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:32.256+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:32.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:32.245+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:32.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:32 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:32.379+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:32.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:32 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:32.435+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:33.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:33 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:33.334+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:33.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:33 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:33.292+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:33.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:33 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:33.292+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.088870+0000 front 2024-04-23T18:47:09.088950+0000 (oldest deadline 2024-04-23T18:47:33.188918+0000) 2024-04-23T18:47:33.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:33 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:33.275+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:33.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:33 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:33.394+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:33.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:33 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:33.468+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:34.263 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:33 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-23T18:47:33.807+0000 7ffba9b54700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.499542+0000 front 2024-04-23T18:47:09.499493+0000 (oldest deadline 2024-04-23T18:47:32.999718+0000) 2024-04-23T18:47:34.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:34 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:34.298+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:34.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:34 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:34.416+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:34.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:34 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:34.501+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:34.788 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:34 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:34.323+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:34.789 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:34 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:34.332+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:34.789 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:34 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:34.305+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:34.789 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:34 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:34.305+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.088870+0000 front 2024-04-23T18:47:09.088950+0000 (oldest deadline 2024-04-23T18:47:33.188918+0000) 2024-04-23T18:47:35.229 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:34 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-23T18:47:34.853+0000 7ffba9b54700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.499542+0000 front 2024-04-23T18:47:09.499493+0000 (oldest deadline 2024-04-23T18:47:32.999718+0000) 2024-04-23T18:47:35.513 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:35 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:35.238+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:35.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:35 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:35.296+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:35.611 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:35 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:35.296+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:35.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:35 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:35.371+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:35.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:35 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:35.371+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:35.789 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:35 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:35.305+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:35.789 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:35 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:35.305+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:35.789 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:35 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:35.315+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:35.789 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:35 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:35.315+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.088870+0000 front 2024-04-23T18:47:09.088950+0000 (oldest deadline 2024-04-23T18:47:33.188918+0000) 2024-04-23T18:47:35.806 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:35 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:35.529+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:36.247 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:35 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4[43578]: 2024-04-23T18:47:35.815+0000 7ffba9b54700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.499542+0000 front 2024-04-23T18:47:09.499493+0000 (oldest deadline 2024-04-23T18:47:32.999718+0000) 2024-04-23T18:47:36.513 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:36 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:36.255+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:36.538 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:36 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:36.307+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:36.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:36 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:36.280+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:36.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:36 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:36.269+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:36.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:36 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:36.269+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.088870+0000 front 2024-04-23T18:47:09.088950+0000 (oldest deadline 2024-04-23T18:47:33.188918+0000) 2024-04-23T18:47:36.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:36 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:36.262+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:36.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:36 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:36.262+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:36.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:36 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:36.415+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:36.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:36 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:36.415+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:36.924 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:36 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:36.530+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:36.925 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:36 smithi148 podman[68418]: 2024-04-23 18:47:36.626214479 +0000 UTC m=+5.582462629 container died cf90c0fe091ee40066b9bb52a8c79fb533048b127a213f115789932edb0663f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T18:47:37.252 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:37 smithi148 podman[68418]: 2024-04-23 18:47:37.025710717 +0000 UTC m=+5.981958863 container cleanup cf90c0fe091ee40066b9bb52a8c79fb533048b127a213f115789932edb0663f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-23T18:47:37.252 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:37 smithi148 bash[68418]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4 2024-04-23T18:47:37.513 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:37 smithi148 podman[68436]: 2024-04-23 18:47:37.28519539 +0000 UTC m=+0.653100824 container remove cf90c0fe091ee40066b9bb52a8c79fb533048b127a213f115789932edb0663f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:47:37.513 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:37 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:37.261+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:37.538 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:37 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:37.263+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:37.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:37 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:37.269+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:37.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:37 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:37.269+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.088870+0000 front 2024-04-23T18:47:09.088950+0000 (oldest deadline 2024-04-23T18:47:33.188918+0000) 2024-04-23T18:47:37.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:37 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:37.251+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:37.540 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:37 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:37.251+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:37.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:37 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:37.270+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:37.611 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:37 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:37.270+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:37.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:37 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:37.388+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:37.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:37 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:37.388+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:37.905 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:37 smithi148 podman[68527]: 2024-04-23 18:47:37.635893269 +0000 UTC m=+0.110059234 container create 1ccce93494d9338a98139f75dee84d7e5265275f26cb57de416773ceb0d7f9cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:47:37.905 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:37 smithi148 podman[68527]: 2024-04-23 18:47:37.543860768 +0000 UTC m=+0.018026757 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:47:37.905 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:37 smithi148 podman[68527]: 2024-04-23 18:47:37.914117817 +0000 UTC m=+0.388283788 container init 1ccce93494d9338a98139f75dee84d7e5265275f26cb57de416773ceb0d7f9cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:47:37.905 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:37 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:37.567+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:37.906 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:37 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:37.567+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:38.203 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:37 smithi148 podman[68527]: 2024-04-23 18:47:37.917557622 +0000 UTC m=+0.391723589 container start 1ccce93494d9338a98139f75dee84d7e5265275f26cb57de416773ceb0d7f9cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:47:38.204 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:37 smithi148 podman[68527]: 2024-04-23 18:47:37.988694323 +0000 UTC m=+0.462860298 container attach 1ccce93494d9338a98139f75dee84d7e5265275f26cb57de416773ceb0d7f9cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:47:38.498 INFO:journalctl@ceph.osd.4.smithi148.stdout:Apr 23 18:47:38 smithi148 podman[68527]: 2024-04-23 18:47:38.21360881 +0000 UTC m=+0.687774780 container died 1ccce93494d9338a98139f75dee84d7e5265275f26cb57de416773ceb0d7f9cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:47:38.498 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:38 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:38.299+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:38.538 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:38 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:38.250+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:38.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:38 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:38.240+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:38.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:38 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:38.240+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.088870+0000 front 2024-04-23T18:47:09.088950+0000 (oldest deadline 2024-04-23T18:47:33.188918+0000) 2024-04-23T18:47:38.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:38 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:38.228+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:38.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:38 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:38.228+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:38.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:38 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:38.277+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:38.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:38 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:38.277+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:38.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:38 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:38.383+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:38.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:38 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:38.383+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:38.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:38 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:38.540+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:38.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:38 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:38.540+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:38.814 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.4.service' 2024-04-23T18:47:38.852 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:47:38.853 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-23T18:47:38.853 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-23T18:47:38.853 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.5 2024-04-23T18:47:39.157 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:38 smithi157 systemd[1]: Stopping Ceph osd.5 for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:47:39.538 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:39 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:39.291+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:39.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:39 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:39.230+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:39.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:39 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:39.230+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:39.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:39 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:39.167+0000 7f7585dd8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:47:39.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:39 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:39.167+0000 7f7585dd8700 -1 osd.5 102 *** Got signal Terminated *** 2024-04-23T18:47:39.540 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:39 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:39.167+0000 7f7585dd8700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:47:39.540 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:39 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:39.254+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:39.540 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:39 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:39.254+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.088870+0000 front 2024-04-23T18:47:09.088950+0000 (oldest deadline 2024-04-23T18:47:33.188918+0000) 2024-04-23T18:47:39.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:39 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:39.250+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:39.611 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:39 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:39.250+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:39.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:39 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:39.396+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:39.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:39 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:39.396+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:39.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:39 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:39.315+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:39.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:39 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:39.492+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:39.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:39 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:39.492+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:40.407 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:40 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:40.292+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:40.408 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:40 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:40.292+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:40.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:40 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:40.274+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:40.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:40 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:40.256+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:40.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:40 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:40.256+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.088870+0000 front 2024-04-23T18:47:09.088950+0000 (oldest deadline 2024-04-23T18:47:33.188918+0000) 2024-04-23T18:47:40.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:40 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:40.270+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:40.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:40 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:40.270+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:40.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:40 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:40.325+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:40.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:40 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:40.513+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:40.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:40 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:40.513+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:40.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:40 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:40.513+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:40.860 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:40 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:40.416+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:40.860 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:40 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:40.416+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:41.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:41 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:41.282+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:41.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:41 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:41.297+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:41.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:41 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:41.297+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:41.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:41 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:41.279+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:41.540 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:41 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:41.279+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.088870+0000 front 2024-04-23T18:47:09.088950+0000 (oldest deadline 2024-04-23T18:47:33.188918+0000) 2024-04-23T18:47:41.540 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:41 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:41.279+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:16.491201+0000 front 2024-04-23T18:47:16.491262+0000 (oldest deadline 2024-04-23T18:47:41.191333+0000) 2024-04-23T18:47:41.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:41 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:41.304+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:41.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:41 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:41.304+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:41.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:41 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:41.304+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.163651+0000 front 2024-04-23T18:47:15.163755+0000 (oldest deadline 2024-04-23T18:47:40.463688+0000) 2024-04-23T18:47:41.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:41 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:41.378+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:41.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:41 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:41.378+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:41.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:41 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:41.378+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:41.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:41 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:41.359+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:41.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:41 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:41.463+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:41.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:41 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:41.463+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:41.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:41 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:41.463+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:42.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:42 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:42.287+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:42.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:42 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:42.291+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:42.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:42 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:42.291+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:42.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:42 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:42.233+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:42.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:42 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:42.233+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.088870+0000 front 2024-04-23T18:47:09.088950+0000 (oldest deadline 2024-04-23T18:47:33.188918+0000) 2024-04-23T18:47:42.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:42 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:42.233+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:16.491201+0000 front 2024-04-23T18:47:16.491262+0000 (oldest deadline 2024-04-23T18:47:41.191333+0000) 2024-04-23T18:47:42.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:42 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:42.288+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:42.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:42 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:42.288+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:42.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:42 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:42.288+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.163651+0000 front 2024-04-23T18:47:15.163755+0000 (oldest deadline 2024-04-23T18:47:40.463688+0000) 2024-04-23T18:47:42.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:42 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:42.344+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:42.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:42 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:42.344+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:42.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:42 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:42.344+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:42.663 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:42 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:42.324+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:42.664 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:42 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:42.443+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:42.664 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:42 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:42.443+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:42.664 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:42 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:42.443+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:43.538 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:43 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:43.292+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:43.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:43 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:43.315+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:43.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:43 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:43.315+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:43.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:43 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:43.315+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:43.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:43 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:43.203+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:03.288087+0000 front 2024-04-23T18:47:03.287803+0000 (oldest deadline 2024-04-23T18:47:24.987874+0000) 2024-04-23T18:47:43.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:43 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:43.203+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.088870+0000 front 2024-04-23T18:47:09.088950+0000 (oldest deadline 2024-04-23T18:47:33.188918+0000) 2024-04-23T18:47:43.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:43 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5[44525]: 2024-04-23T18:47:43.203+0000 7f75823cc700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:16.491201+0000 front 2024-04-23T18:47:16.491262+0000 (oldest deadline 2024-04-23T18:47:41.191333+0000) 2024-04-23T18:47:43.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:43 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:43.319+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:43.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:43 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:43.319+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:43.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:43 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:43.319+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.163651+0000 front 2024-04-23T18:47:15.163755+0000 (oldest deadline 2024-04-23T18:47:40.463688+0000) 2024-04-23T18:47:43.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:43 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:43.331+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:43.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:43 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:43.331+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:43.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:43 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:43.331+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:43.680 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:43 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:43.458+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:43.680 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:43 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:43.458+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:43.680 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:43 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:43.458+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:43.681 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:43 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:43.360+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:44.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:44 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:44.319+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:44.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:44 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:44.319+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:44.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:44 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:44.319+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.163651+0000 front 2024-04-23T18:47:15.163755+0000 (oldest deadline 2024-04-23T18:47:40.463688+0000) 2024-04-23T18:47:44.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:44 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:44.326+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:44.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:44 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:44.326+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:44.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:44 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:44.326+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:44.696 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:44 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:44.327+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:44.696 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:44 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:44.328+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:44.696 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:44 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:44.328+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:44.696 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:44 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:44.328+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:44.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:44 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:44.356+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:44.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:44 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:44.356+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:44.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:44 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:44.462+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:44.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:44 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:44.462+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:44.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:44 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:44.462+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:45.017 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:44 smithi157 podman[69450]: 2024-04-23 18:47:44.705744238 +0000 UTC m=+5.653652832 container died e3d0602e146f75d4ee57c35cb58c76479c5942068bc02ff64f0d04aacc9a2029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:47:45.275 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:45 smithi157 podman[69450]: 2024-04-23 18:47:45.126008328 +0000 UTC m=+6.073916918 container cleanup e3d0602e146f75d4ee57c35cb58c76479c5942068bc02ff64f0d04aacc9a2029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-23T18:47:45.276 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:45 smithi157 bash[69450]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5 2024-04-23T18:47:45.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:45 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:45.304+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:45.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:45 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:45.304+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:45.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:45 smithi157 podman[69469]: 2024-04-23 18:47:45.360832987 +0000 UTC m=+0.649860426 container remove e3d0602e146f75d4ee57c35cb58c76479c5942068bc02ff64f0d04aacc9a2029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-23T18:47:45.540 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:45 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:45.284+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:45.540 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:45 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:45.284+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:45.540 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:45 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:45.284+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:45.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:45 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:45.280+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:45.611 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:45 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:45.280+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:45.611 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:45 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:45.280+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.163651+0000 front 2024-04-23T18:47:15.163755+0000 (oldest deadline 2024-04-23T18:47:40.463688+0000) 2024-04-23T18:47:45.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:45 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:45.280+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:45.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:45 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:45.280+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:45.612 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:45 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:45.280+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:45.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:45 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:45.396+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:45.764 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:45 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:45.396+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:45.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:45 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:45.466+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:45.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:45 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:45.466+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:45.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:45 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:45.466+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:45.947 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:45 smithi157 podman[69561]: 2024-04-23 18:47:45.585866195 +0000 UTC m=+0.017981693 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:47:45.947 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:45 smithi157 podman[69561]: 2024-04-23 18:47:45.694999047 +0000 UTC m=+0.127114579 container create a19ea322ed698f2a5e0d9bbdd322b44f0df092a4f8d86d52570fa65f80e00059 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-23T18:47:45.947 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:45 smithi157 podman[69561]: 2024-04-23 18:47:45.95636945 +0000 UTC m=+0.388484941 container init a19ea322ed698f2a5e0d9bbdd322b44f0df092a4f8d86d52570fa65f80e00059 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-23T18:47:46.245 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:45 smithi157 podman[69561]: 2024-04-23 18:47:45.960116921 +0000 UTC m=+0.392232418 container start a19ea322ed698f2a5e0d9bbdd322b44f0df092a4f8d86d52570fa65f80e00059 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-23T18:47:46.245 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:46 smithi157 podman[69561]: 2024-04-23 18:47:46.056114505 +0000 UTC m=+0.488230004 container attach a19ea322ed698f2a5e0d9bbdd322b44f0df092a4f8d86d52570fa65f80e00059 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-23T18:47:46.538 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:46 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:46.295+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:46.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:46 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:46.295+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:46.539 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 23 18:47:46 smithi157 podman[69561]: 2024-04-23 18:47:46.254607242 +0000 UTC m=+0.686722729 container died a19ea322ed698f2a5e0d9bbdd322b44f0df092a4f8d86d52570fa65f80e00059 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-23T18:47:46.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:46 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:46.259+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:46.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:46 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:46.259+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:46.540 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:46 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:46.259+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:46.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:46 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:46.313+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:46.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:46 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:46.313+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:46.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:46 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:46.313+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.163651+0000 front 2024-04-23T18:47:15.163755+0000 (oldest deadline 2024-04-23T18:47:40.463688+0000) 2024-04-23T18:47:46.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:46 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:46.330+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:46.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:46 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:46.330+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:46.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:46 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:46.330+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:46.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:46 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:46.363+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:46.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:46 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:46.363+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:46.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:46 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:46.419+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:46.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:46 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:46.419+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:46.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:46 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:46.419+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:46.860 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.5.service' 2024-04-23T18:47:46.899 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:47:46.899 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-23T18:47:46.900 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-23T18:47:46.900 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.6 2024-04-23T18:47:47.216 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:46 smithi132 systemd[1]: Stopping Ceph osd.6 for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:47:47.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:47.261+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:47.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:47.261+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:47.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:47.241+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:47.540 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:47.241+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:47.540 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:47 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:47.241+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:47.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:47.321+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:47.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:47.321+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:47.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:47.321+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:47.611 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:47.225+0000 7f3b68631700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:47:47.611 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:47.225+0000 7f3b68631700 -1 osd.6 102 *** Got signal Terminated *** 2024-04-23T18:47:47.612 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:47.225+0000 7f3b68631700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:47:47.612 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:47.301+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:47.612 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:47.301+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:47.612 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:47 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:47.301+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.163651+0000 front 2024-04-23T18:47:15.163755+0000 (oldest deadline 2024-04-23T18:47:40.463688+0000) 2024-04-23T18:47:47.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:47.323+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:47.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:47.323+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:47.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:47.420+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:47.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:47.420+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:47.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:47 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:47.420+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:48.538 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:48 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:48.224+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:48.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:48 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:48.224+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:48.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:48 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:48.269+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:48.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:48 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:48.269+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:48.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:48 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:48.269+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:48.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:48 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:48.314+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:48.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:48 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:48.314+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:48.611 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:48 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:48.314+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.163651+0000 front 2024-04-23T18:47:15.163755+0000 (oldest deadline 2024-04-23T18:47:40.463688+0000) 2024-04-23T18:47:48.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:48 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:48.330+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:48.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:48 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:48.330+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:48.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:48 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:48.330+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:48.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:48 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:48.339+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:48.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:48 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:48.339+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:48.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:48 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:48.453+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:48.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:48 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:48.453+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:48.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:48 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:48.453+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:49.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:49 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:49.208+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:49.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:49 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:49.208+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:49.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:49 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:49.208+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:47:49.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:49 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:49.317+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:49.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:49 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:49.317+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:49.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:49 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:49.317+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:49.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:49 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:49.292+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:49.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:49 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:49.292+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:49.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:49 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:49.292+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:49.611 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:49 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:49.287+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:49.611 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:49 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:49.287+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:49.611 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:49 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:49.287+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.163651+0000 front 2024-04-23T18:47:15.163755+0000 (oldest deadline 2024-04-23T18:47:40.463688+0000) 2024-04-23T18:47:49.611 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:49 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:49.287+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:25.664961+0000 front 2024-04-23T18:47:25.665040+0000 (oldest deadline 2024-04-23T18:47:48.565012+0000) 2024-04-23T18:47:49.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:49 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:49.386+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:49.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:49 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:49.386+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:49.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:49 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:49.386+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.838093+0000 front 2024-04-23T18:47:23.838137+0000 (oldest deadline 2024-04-23T18:47:48.538141+0000) 2024-04-23T18:47:49.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:49 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:49.486+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:49.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:49 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:49.486+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:49.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:49 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:49.486+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:49.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:49 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:49.486+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:47:50.538 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:50 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:50.222+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:50.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:50 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:50.222+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:50.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:50 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:50.222+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:47:50.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:50 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:50.326+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:50.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:50 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:50.326+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:50.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:50 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:50.326+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:50.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:50 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:50.275+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:50.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:50 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:50.275+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:50.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:50 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:50.275+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.163651+0000 front 2024-04-23T18:47:15.163755+0000 (oldest deadline 2024-04-23T18:47:40.463688+0000) 2024-04-23T18:47:50.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:50 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:50.275+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:25.664961+0000 front 2024-04-23T18:47:25.665040+0000 (oldest deadline 2024-04-23T18:47:48.565012+0000) 2024-04-23T18:47:50.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:50 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:50.275+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:50.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:50 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:50.275+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:50.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:50 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:50.275+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:50.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:50 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:50.386+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:50.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:50 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:50.386+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:50.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:50 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:50.386+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.838093+0000 front 2024-04-23T18:47:23.838137+0000 (oldest deadline 2024-04-23T18:47:48.538141+0000) 2024-04-23T18:47:50.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:50 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:50.491+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:50.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:50 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:50.491+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:50.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:50 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:50.491+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:50.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:50 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:50.491+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:47:51.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:51 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:51.217+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:51.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:51 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:51.217+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:51.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:51 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:51.217+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:47:51.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:51 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:51.336+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:51.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:51 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:51.336+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:51.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:51 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:51.336+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:51.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:51 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:51.263+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:01.661855+0000 front 2024-04-23T18:47:01.662028+0000 (oldest deadline 2024-04-23T18:47:26.962005+0000) 2024-04-23T18:47:51.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:51 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:51.263+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.262973+0000 front 2024-04-23T18:47:09.263028+0000 (oldest deadline 2024-04-23T18:47:35.163172+0000) 2024-04-23T18:47:51.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:51 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:51.263+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.163651+0000 front 2024-04-23T18:47:15.163755+0000 (oldest deadline 2024-04-23T18:47:40.463688+0000) 2024-04-23T18:47:51.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:51 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6[57303]: 2024-04-23T18:47:51.263+0000 7f3b60c1d700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:25.664961+0000 front 2024-04-23T18:47:25.665040+0000 (oldest deadline 2024-04-23T18:47:48.565012+0000) 2024-04-23T18:47:51.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:51 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:51.249+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:51.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:51 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:51.249+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:51.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:51 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:51.249+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:51.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:51 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:51.249+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:47:51.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:51 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:51.408+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:51.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:51 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:51.408+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:51.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:51 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:51.408+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.838093+0000 front 2024-04-23T18:47:23.838137+0000 (oldest deadline 2024-04-23T18:47:48.538141+0000) 2024-04-23T18:47:51.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:51 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:51.477+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:51.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:51 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:51.477+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:51.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:51 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:51.477+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:51.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:51 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:51.477+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:47:52.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:52 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:52.233+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:52.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:52 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:52.233+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:52.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:52 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:52.233+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:47:52.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:52 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:52.288+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:52.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:52 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:52.288+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:52.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:52 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:52.288+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:52.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:52 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:52.247+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:52.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:52 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:52.247+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:52.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:52 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:52.247+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:52.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:52 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:52.247+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:47:52.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:52 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:52.371+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:52.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:52 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:52.371+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:52.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:52 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:52.371+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.838093+0000 front 2024-04-23T18:47:23.838137+0000 (oldest deadline 2024-04-23T18:47:48.538141+0000) 2024-04-23T18:47:52.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:52 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:52.468+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:52.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:52 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:52.468+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:52.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:52 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:52.468+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:52.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:52 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:52.468+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:47:52.976 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:52 smithi132 podman[89762]: 2024-04-23 18:47:52.684803673 +0000 UTC m=+5.580782409 container died 3c3d6b3ad09f22d5c393c99f786844f3fd3fe23318762c6f893b7c353bb8f217 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:47:53.272 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:53 smithi132 podman[89762]: 2024-04-23 18:47:53.09347141 +0000 UTC m=+5.989450141 container cleanup 3c3d6b3ad09f22d5c393c99f786844f3fd3fe23318762c6f893b7c353bb8f217 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-23T18:47:53.272 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:53 smithi132 bash[89762]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6 2024-04-23T18:47:53.538 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:53 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:53.264+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:53.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:53 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:53.264+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:53.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:53 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:53.264+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:47:53.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:53 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:53.321+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:53.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:53 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:53.321+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:53.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:53 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:53.321+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:53.610 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:53 smithi132 podman[89779]: 2024-04-23 18:47:53.311210743 +0000 UTC m=+0.621644574 container remove 3c3d6b3ad09f22d5c393c99f786844f3fd3fe23318762c6f893b7c353bb8f217 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:47:53.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:53 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:53.281+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:53.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:53 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:53.281+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:53.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:53 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:53.281+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:53.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:53 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:53.281+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:47:53.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:53 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:53.418+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:53.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:53 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:53.418+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:53.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:53 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:53.418+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.838093+0000 front 2024-04-23T18:47:23.838137+0000 (oldest deadline 2024-04-23T18:47:48.538141+0000) 2024-04-23T18:47:53.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:53 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:53.485+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:53.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:53 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:53.485+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:53.765 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:53 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:53.485+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:53.765 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:53 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:53.485+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:47:54.005 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:53 smithi132 podman[89872]: 2024-04-23 18:47:53.753728046 +0000 UTC m=+0.109908207 container create 9e522f35e9bde9e4ba438ff926ff39732e4d375858d03c6dd1e45a40c1b351ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-23T18:47:54.005 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:53 smithi132 podman[89872]: 2024-04-23 18:47:53.661946107 +0000 UTC m=+0.018126280 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:47:54.259 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:54 smithi132 podman[89872]: 2024-04-23 18:47:54.014785159 +0000 UTC m=+0.370965324 container init 9e522f35e9bde9e4ba438ff926ff39732e4d375858d03c6dd1e45a40c1b351ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T18:47:54.260 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:54 smithi132 podman[89872]: 2024-04-23 18:47:54.017915416 +0000 UTC m=+0.374095585 container start 9e522f35e9bde9e4ba438ff926ff39732e4d375858d03c6dd1e45a40c1b351ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T18:47:54.260 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:54 smithi132 podman[89872]: 2024-04-23 18:47:54.106427497 +0000 UTC m=+0.462607673 container attach 9e522f35e9bde9e4ba438ff926ff39732e4d375858d03c6dd1e45a40c1b351ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-23T18:47:54.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:54 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:54.301+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:54.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:54 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:54.301+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:54.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:54 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:54.301+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:54.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:54 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:54.256+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:54.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:54 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:54.256+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:54.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:54 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:54.256+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:47:54.609 INFO:journalctl@ceph.osd.6.smithi132.stdout:Apr 23 18:47:54 smithi132 podman[89872]: 2024-04-23 18:47:54.314246361 +0000 UTC m=+0.670426514 container died 9e522f35e9bde9e4ba438ff926ff39732e4d375858d03c6dd1e45a40c1b351ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-23T18:47:54.609 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:54 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:54.268+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:54.609 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:54 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:54.268+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:54.609 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:54 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:54.268+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:54.609 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:54 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:54.268+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:47:54.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:54 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:54.403+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:54.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:54 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:54.403+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:54.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:54 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:54.403+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.838093+0000 front 2024-04-23T18:47:23.838137+0000 (oldest deadline 2024-04-23T18:47:48.538141+0000) 2024-04-23T18:47:54.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:54 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:54.475+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:54.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:54 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:54.475+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:54.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:54 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:54.475+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:54.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:54 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:54.475+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:47:54.888 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.6.service' 2024-04-23T18:47:54.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:47:54.925 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-23T18:47:54.926 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-23T18:47:54.926 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.7 2024-04-23T18:47:55.235 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:54 smithi148 systemd[1]: Stopping Ceph osd.7 for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:47:55.513 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:55.474+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:55.513 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:55.474+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:55.513 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:55.474+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:55.513 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:55.474+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:47:55.514 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:55.244+0000 7fbf7ec93700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:47:55.514 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:55.244+0000 7fbf7ec93700 -1 osd.7 102 *** Got signal Terminated *** 2024-04-23T18:47:55.514 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:55.244+0000 7fbf7ec93700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:47:55.514 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:55.396+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:55.514 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:55.396+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:55.514 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:55 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:55.396+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.838093+0000 front 2024-04-23T18:47:23.838137+0000 (oldest deadline 2024-04-23T18:47:48.538141+0000) 2024-04-23T18:47:55.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:55 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:55.293+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:55.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:55 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:55.293+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:55.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:55 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:55.293+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:55.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:55 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:55.293+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:47:55.789 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:55 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:55.344+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:55.789 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:55 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:55.344+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:55.789 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:55 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:55.344+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:55.790 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:55 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:55.306+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:55.790 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:55 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:55.306+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:55.790 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:55 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:55.306+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:47:56.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:56 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:56.274+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:56.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:56 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:56.274+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:56.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:56 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:56.274+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:47:56.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:56 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:56.308+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:56.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:56 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:56.308+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:56.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:56 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:56.308+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:56.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:56 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:56.300+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:56.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:56 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:56.300+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:56.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:56 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:56.300+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:56.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:56 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:56.300+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:47:56.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:56 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:56.357+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:56.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:56 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:56.357+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:56.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:56 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:56.357+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.838093+0000 front 2024-04-23T18:47:23.838137+0000 (oldest deadline 2024-04-23T18:47:48.538141+0000) 2024-04-23T18:47:56.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:56 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:56.494+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:56.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:56 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:56.494+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:56.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:56 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:56.494+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:56.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:56 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:56.494+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:47:57.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:57 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:57.305+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:57.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:57 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:57.305+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:57.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:57 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:57.305+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:57.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:57 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:57.229+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:57.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:57 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:57.229+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:57.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:57 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:57.229+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:47:57.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:57 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:57.285+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:57.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:57 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:57.285+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:57.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:57 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:57.285+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:57.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:57 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:57.285+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:47:57.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:57 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:57.366+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:57.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:57 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:57.366+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:57.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:57 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:57.366+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.838093+0000 front 2024-04-23T18:47:23.838137+0000 (oldest deadline 2024-04-23T18:47:48.538141+0000) 2024-04-23T18:47:57.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:57 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:57.366+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.439207+0000 front 2024-04-23T18:47:34.439248+0000 (oldest deadline 2024-04-23T18:47:57.339330+0000) 2024-04-23T18:47:57.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:57 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:57.532+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:57.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:57 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:57.532+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:57.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:57 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:57.532+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:57.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:57 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:57.532+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:47:58.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:58 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:58.217+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:58.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:58 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:58.217+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:58.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:58 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:58.217+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:47:58.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:58 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:58.309+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:58.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:58 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:58.309+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:58.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:58 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:58.309+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:58.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:58.276+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:58.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:58.277+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:58.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:58.277+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:58.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:58 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:58.277+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:47:58.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:58 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:58.339+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:58.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:58 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:58.339+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:58.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:58 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:58.339+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.838093+0000 front 2024-04-23T18:47:23.838137+0000 (oldest deadline 2024-04-23T18:47:48.538141+0000) 2024-04-23T18:47:58.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:58 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:58.339+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.439207+0000 front 2024-04-23T18:47:34.439248+0000 (oldest deadline 2024-04-23T18:47:57.339330+0000) 2024-04-23T18:47:58.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:58 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:58.564+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:58.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:58 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:58.564+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:58.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:58 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:58.564+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:58.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:58 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:58.564+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:47:59.528 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:59 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:59.279+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:47:59.528 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:59 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:59.279+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:47:59.528 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:59 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:59.279+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:47:59.528 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:47:59 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:47:59.279+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:47:59.538 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:59 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:59.203+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:47:59.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:59 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:59.203+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:47:59.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:47:59 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:47:59.203+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:47:59.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:59 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:59.298+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:47:59.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:59 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:59.298+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:47:59.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:59 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:59.298+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:47:59.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:47:59 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:47:59.298+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:47:59.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:59 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:59.340+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.936184+0000 front 2024-04-23T18:47:10.936158+0000 (oldest deadline 2024-04-23T18:47:35.036274+0000) 2024-04-23T18:47:59.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:59 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:59.340+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.737334+0000 front 2024-04-23T18:47:19.737292+0000 (oldest deadline 2024-04-23T18:47:43.837507+0000) 2024-04-23T18:47:59.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:59 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:59.340+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.838093+0000 front 2024-04-23T18:47:23.838137+0000 (oldest deadline 2024-04-23T18:47:48.538141+0000) 2024-04-23T18:47:59.763 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:47:59 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7[50418]: 2024-04-23T18:47:59.340+0000 7fbf7727f700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.439207+0000 front 2024-04-23T18:47:34.439248+0000 (oldest deadline 2024-04-23T18:47:57.339330+0000) 2024-04-23T18:47:59.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:59 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:59.526+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:47:59.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:59 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:59.526+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:47:59.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:59 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:59.526+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:47:59.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:47:59 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:47:59.527+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:00.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:00 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:00.322+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:00.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:00 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:00.322+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:00.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:00 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:00.322+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:00.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:00 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:00.322+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:00.541 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:00 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:00.227+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:48:00.541 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:00 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:00.227+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:48:00.541 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:00 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:00.227+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:48:00.541 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:00 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:00.227+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:36.053588+0000 front 2024-04-23T18:47:36.053603+0000 (oldest deadline 2024-04-23T18:47:59.553366+0000) 2024-04-23T18:48:00.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:00.271+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:00.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:00.271+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:00.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:00.271+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:00.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:00 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:00.271+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:01.002 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:48:00 smithi148 podman[68644]: 2024-04-23 18:48:00.72430165 +0000 UTC m=+5.602783213 container died 6d3988c0c5928cbfdc1c66dab4339cd2ae2eb970df65196db7923a181e9dfe87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-23T18:48:01.003 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:00 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:00.531+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:01.003 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:00 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:00.531+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:01.003 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:00 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:00.531+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:01.003 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:00 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:00.531+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:01.263 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:48:01 smithi148 podman[68644]: 2024-04-23 18:48:01.152835801 +0000 UTC m=+6.031317361 container cleanup 6d3988c0c5928cbfdc1c66dab4339cd2ae2eb970df65196db7923a181e9dfe87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-23T18:48:01.263 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:48:01 smithi148 bash[68644]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7 2024-04-23T18:48:01.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:01 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:01.237+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:48:01.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:01 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:01.237+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:48:01.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:01 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:01.237+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:48:01.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:01 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:01.237+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:36.053588+0000 front 2024-04-23T18:47:36.053603+0000 (oldest deadline 2024-04-23T18:47:59.553366+0000) 2024-04-23T18:48:01.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:01 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:01.291+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:01.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:01 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:01.291+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:01.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:01 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:01.291+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:01.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:01 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:01.291+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:01.615 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:01 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:01.303+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:01.616 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:01 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:01.303+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:01.616 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:01 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:01.303+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:01.616 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:01 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:01.303+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:01.762 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:48:01 smithi148 podman[68661]: 2024-04-23 18:48:01.370746562 +0000 UTC m=+0.643810912 container remove 6d3988c0c5928cbfdc1c66dab4339cd2ae2eb970df65196db7923a181e9dfe87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-23T18:48:01.765 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:01 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:01.534+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:01.765 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:01 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:01.534+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:01.765 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:01 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:01.534+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:01.765 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:01 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:01.534+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:01.765 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:01 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:01.534+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:02.013 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:48:01 smithi148 podman[68749]: 2024-04-23 18:48:01.674922765 +0000 UTC m=+0.018151330 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:48:02.013 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:48:01 smithi148 podman[68749]: 2024-04-23 18:48:01.788389509 +0000 UTC m=+0.131618051 container create 466849d32cb4404b6c48bc3be3cb765a6b39ee343d887a14b9dd7ddd6174c365 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-23T18:48:02.288 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:48:02 smithi148 podman[68749]: 2024-04-23 18:48:02.023976964 +0000 UTC m=+0.367205506 container init 466849d32cb4404b6c48bc3be3cb765a6b39ee343d887a14b9dd7ddd6174c365 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-23T18:48:02.288 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:48:02 smithi148 podman[68749]: 2024-04-23 18:48:02.02821604 +0000 UTC m=+0.371444584 container start 466849d32cb4404b6c48bc3be3cb765a6b39ee343d887a14b9dd7ddd6174c365 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-23T18:48:02.288 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:48:02 smithi148 podman[68749]: 2024-04-23 18:48:02.107462126 +0000 UTC m=+0.450690671 container attach 466849d32cb4404b6c48bc3be3cb765a6b39ee343d887a14b9dd7ddd6174c365 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-23T18:48:02.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:02 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:02.283+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:48:02.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:02 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:02.283+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:48:02.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:02 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:02.283+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:48:02.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:02 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:02.283+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:36.053588+0000 front 2024-04-23T18:47:36.053603+0000 (oldest deadline 2024-04-23T18:47:59.553366+0000) 2024-04-23T18:48:02.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:02 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:02.250+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:02.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:02 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:02.250+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:02.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:02 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:02.250+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:02.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:02 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:02.250+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:02.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Apr 23 18:48:02 smithi148 podman[68749]: 2024-04-23 18:48:02.33188356 +0000 UTC m=+0.675112112 container died 466849d32cb4404b6c48bc3be3cb765a6b39ee343d887a14b9dd7ddd6174c365 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-23T18:48:02.609 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:02 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:02.539+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:02.609 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:02 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:02.539+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:02.609 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:02 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:02.540+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:02.609 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:02 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:02.540+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:02.609 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:02 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:02.540+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:02.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:02.274+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:02.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:02.274+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:02.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:02.274+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:02.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:02 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:02.274+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:02.905 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.7.service' 2024-04-23T18:48:02.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:48:02.942 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-23T18:48:02.942 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-23T18:48:02.942 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.8 2024-04-23T18:48:03.244 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:03 smithi157 systemd[1]: Stopping Ceph osd.8 for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:48:03.245 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:03 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:03.253+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:48:03.245 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:03 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:03.253+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:48:03.245 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:03 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:03.202+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:03.245 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:03 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:03.202+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:03.245 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:03 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:03.202+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:03.245 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:03 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:03.202+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:03.538 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:03 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:03.253+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:48:03.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:03 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:03.253+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:36.053588+0000 front 2024-04-23T18:47:36.053603+0000 (oldest deadline 2024-04-23T18:47:59.553366+0000) 2024-04-23T18:48:03.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:03 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:03.281+0000 7f53d6727700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:48:03.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:03 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:03.281+0000 7f53d6727700 -1 osd.8 102 *** Got signal Terminated *** 2024-04-23T18:48:03.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:03 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:03.281+0000 7f53d6727700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:48:03.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:03 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:03.279+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:03.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:03 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:03.279+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:03.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:03 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:03.279+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:03.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:03 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:03.279+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:03.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:03 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:03.520+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:03.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:03 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:03.520+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:03.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:03 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:03.520+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:03.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:03 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:03.520+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:03.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:03 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:03.520+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:04.538 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:04.277+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:48:04.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:04.277+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:48:04.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:04.277+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:48:04.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:04.277+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:36.053588+0000 front 2024-04-23T18:47:36.053603+0000 (oldest deadline 2024-04-23T18:47:59.553366+0000) 2024-04-23T18:48:04.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:04.164+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:04.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:04.164+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:04.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:04.164+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:04.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:04 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:04.164+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:04.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:04 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:04.312+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:04.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:04 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:04.312+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:04.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:04 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:04.312+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:04.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:04 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:04.312+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:05.013 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:04 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:04.548+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:05.013 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:04 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:04.548+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:05.013 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:04 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:04.548+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:05.013 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:04 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:04.548+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:05.013 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:04 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:04.548+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:05.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:05 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:05.180+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:05.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:05 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:05.180+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:05.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:05 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:05.180+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:05.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:05 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:05.180+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:05.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:05 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:05.180+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:05.540 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:05 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:05.302+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:48:05.540 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:05 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:05.302+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:48:05.540 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:05 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:05.302+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:48:05.540 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:05 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:05.302+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:36.053588+0000 front 2024-04-23T18:47:36.053603+0000 (oldest deadline 2024-04-23T18:47:59.553366+0000) 2024-04-23T18:48:05.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:05 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:05.275+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:05.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:05 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:05.275+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:05.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:05 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:05.275+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:05.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:05 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:05.275+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:06.013 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:05 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:05.537+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:06.013 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:05 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:05.537+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:06.013 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:05 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:05.537+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:06.013 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:05 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:05.537+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:06.013 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:05 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:05.537+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:06.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:06.318+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:48:06.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:06.318+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:48:06.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:06.318+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:48:06.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:06.318+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:36.053588+0000 front 2024-04-23T18:47:36.053603+0000 (oldest deadline 2024-04-23T18:47:59.553366+0000) 2024-04-23T18:48:06.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:06.220+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:06.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:06.220+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:06.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:06.220+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:06.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:06.220+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:06.540 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:06 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:06.220+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:06.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:06 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:06.227+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:06.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:06 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:06.227+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:06.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:06 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:06.227+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:06.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:06 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:06.227+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:06.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:06 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:06.227+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:40.787093+0000 front 2024-04-23T18:47:40.787132+0000 (oldest deadline 2024-04-23T18:48:06.087168+0000) 2024-04-23T18:48:06.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:06 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:06.505+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:06.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:06 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:06.505+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:06.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:06 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:06.505+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:06.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:06 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:06.505+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:06.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:06 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:06.505+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:07.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:07 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:07.343+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:09.248497+0000 front 2024-04-23T18:47:09.248436+0000 (oldest deadline 2024-04-23T18:47:33.348683+0000) 2024-04-23T18:48:07.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:07 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:07.343+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:19.651078+0000 front 2024-04-23T18:47:19.651037+0000 (oldest deadline 2024-04-23T18:47:44.351151+0000) 2024-04-23T18:48:07.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:07 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:07.343+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:24.351774+0000 front 2024-04-23T18:47:24.351892+0000 (oldest deadline 2024-04-23T18:47:49.051587+0000) 2024-04-23T18:48:07.539 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:07 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8[51239]: 2024-04-23T18:48:07.343+0000 7f53d2d1b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:36.053588+0000 front 2024-04-23T18:47:36.053603+0000 (oldest deadline 2024-04-23T18:47:59.553366+0000) 2024-04-23T18:48:07.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:07 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:07.203+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:07.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:07 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:07.203+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:07.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:07 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:07.203+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:07.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:07 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:07.203+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:07.540 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:07 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:07.203+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:07.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:07 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:07.178+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:07.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:07 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:07.178+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:07.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:07 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:07.178+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:07.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:07 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:07.178+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:07.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:07 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:07.178+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:40.787093+0000 front 2024-04-23T18:47:40.787132+0000 (oldest deadline 2024-04-23T18:48:06.087168+0000) 2024-04-23T18:48:07.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:07 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:07.468+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:07.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:07 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:07.468+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:07.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:07 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:07.468+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:07.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:07 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:07.468+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:07.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:07 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:07.468+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:07.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:07 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:07.468+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:08.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:08 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:08.203+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:08.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:08 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:08.203+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:08.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:08 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:08.203+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:08.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:08 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:08.203+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:08.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:08 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:08.203+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:08.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:08 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:08.132+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:08.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:08 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:08.132+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:08.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:08 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:08.132+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:08.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:08 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:08.132+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:08.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:08 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:08.132+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:40.787093+0000 front 2024-04-23T18:47:40.787132+0000 (oldest deadline 2024-04-23T18:48:06.087168+0000) 2024-04-23T18:48:08.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:08 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:08.427+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:08.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:08 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:08.427+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:08.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:08 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:08.427+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:08.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:08 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:08.427+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:08.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:08 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:08.427+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:08.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:08 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:08.427+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:08.994 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:08 smithi157 podman[69666]: 2024-04-23 18:48:08.695169221 +0000 UTC m=+5.547420154 container died b96f5af7c11a0e7c35f862cc9448f57080f78518ed664cf29935fb01a7d0cb8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-23T18:48:09.288 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:09 smithi157 podman[69666]: 2024-04-23 18:48:09.086824192 +0000 UTC m=+5.939075124 container cleanup b96f5af7c11a0e7c35f862cc9448f57080f78518ed664cf29935fb01a7d0cb8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:48:09.289 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:09 smithi157 bash[69666]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8 2024-04-23T18:48:09.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:09.198+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:09.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:09.198+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:09.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:09.198+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:09.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:09.198+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:09.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:09 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:09.198+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:09.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:09 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:09.141+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:09.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:09 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:09.141+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:09.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:09 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:09.141+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:09.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:09 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:09.141+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:09.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:09 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:09.141+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:40.787093+0000 front 2024-04-23T18:47:40.787132+0000 (oldest deadline 2024-04-23T18:48:06.087168+0000) 2024-04-23T18:48:09.746 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:09 smithi157 podman[69686]: 2024-04-23 18:48:09.405397003 +0000 UTC m=+0.708817256 container remove b96f5af7c11a0e7c35f862cc9448f57080f78518ed664cf29935fb01a7d0cb8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:48:09.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:09 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:09.407+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:09.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:09 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:09.407+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:09.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:09 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:09.407+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:09.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:09 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:09.407+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:09.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:09 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:09.407+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:09.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:09 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:09.407+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:10.016 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:09 smithi157 podman[69776]: 2024-04-23 18:48:09.756266308 +0000 UTC m=+0.109414725 container create f627ff000dd443ea1ff39814a90a1ccf76ec9e4b762142de47239d9fb41fd6e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:48:10.016 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:09 smithi157 podman[69776]: 2024-04-23 18:48:09.665300999 +0000 UTC m=+0.018449389 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:48:10.016 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:10 smithi157 podman[69776]: 2024-04-23 18:48:10.025745965 +0000 UTC m=+0.378894348 container init f627ff000dd443ea1ff39814a90a1ccf76ec9e4b762142de47239d9fb41fd6e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-23T18:48:10.288 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:10 smithi157 podman[69776]: 2024-04-23 18:48:10.02922413 +0000 UTC m=+0.382372512 container start f627ff000dd443ea1ff39814a90a1ccf76ec9e4b762142de47239d9fb41fd6e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:48:10.289 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:10 smithi157 podman[69776]: 2024-04-23 18:48:10.108658141 +0000 UTC m=+0.461806531 container attach f627ff000dd443ea1ff39814a90a1ccf76ec9e4b762142de47239d9fb41fd6e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-23T18:48:10.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:10 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:10.168+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:10.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:10 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:10.168+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:10.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:10 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:10.168+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:10.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:10 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:10.168+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:10.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:10 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:10.168+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:10.584 INFO:journalctl@ceph.osd.8.smithi157.stdout:Apr 23 18:48:10 smithi157 podman[69776]: 2024-04-23 18:48:10.32290007 +0000 UTC m=+0.676048445 container died f627ff000dd443ea1ff39814a90a1ccf76ec9e4b762142de47239d9fb41fd6e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-23T18:48:10.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:10 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:10.173+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:10.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:10 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:10.173+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:10.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:10 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:10.173+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:10.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:10 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:10.173+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:10.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:10 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:10.173+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:40.787093+0000 front 2024-04-23T18:47:40.787132+0000 (oldest deadline 2024-04-23T18:48:06.087168+0000) 2024-04-23T18:48:10.765 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:10 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:10.414+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:10.765 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:10 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:10.414+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:10.766 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:10 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:10.414+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:10.766 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:10 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:10.414+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:10.766 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:10 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:10.414+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:10.766 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:10 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:10.414+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:10.938 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.8.service' 2024-04-23T18:48:10.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:48:10.977 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-23T18:48:10.977 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-23T18:48:10.978 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.9 2024-04-23T18:48:11.288 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:11 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:11.171+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:11.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:11 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:11.171+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:11.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:11 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:11.171+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:11.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:11 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:11.171+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:11.289 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:11 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:11.171+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:11.360 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:11 smithi132 systemd[1]: Stopping Ceph osd.9 for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:48:11.360 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:11 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:11.180+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:11.360 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:11 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:11.180+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:11.361 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:11 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:11.180+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:11.361 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:11 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:11.180+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:11.361 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:11 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:11.180+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:40.787093+0000 front 2024-04-23T18:47:40.787132+0000 (oldest deadline 2024-04-23T18:48:06.087168+0000) 2024-04-23T18:48:11.361 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:11 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:11.262+0000 7f91ee51a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:48:11.361 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:11 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:11.262+0000 7f91ee51a700 -1 osd.9 102 *** Got signal Terminated *** 2024-04-23T18:48:11.361 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:11 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:11.262+0000 7f91ee51a700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:48:11.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:11 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:11.379+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:11.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:11 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:11.379+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:11.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:11 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:11.379+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:11.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:11 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:11.379+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:11.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:11 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:11.379+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:11.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:11 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:11.379+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:12.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:12 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:12.191+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:12.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:12 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:12.191+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:12.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:12 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:12.191+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:12.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:12 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:12.191+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:12.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:12 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:12.191+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:12.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:12 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:12.184+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:12.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:12 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:12.184+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:12.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:12 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:12.184+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:12.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:12 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:12.184+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:12.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:12 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:12.184+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:40.787093+0000 front 2024-04-23T18:47:40.787132+0000 (oldest deadline 2024-04-23T18:48:06.087168+0000) 2024-04-23T18:48:12.664 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:12 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:12.395+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:12.665 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:12 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:12.395+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:12.665 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:12 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:12.395+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:12.665 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:12 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:12.395+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:12.665 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:12 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:12.395+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:12.665 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:12 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:12.395+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:13.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:13 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:13.169+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:13.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:13 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:13.169+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:13.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:13 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:13.169+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:13.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:13 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:13.169+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:13.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:13 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:13.169+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:13.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:13 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:13.169+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:13.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:13 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:13.144+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:13.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:13 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:13.144+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:13.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:13 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:13.144+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:13.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:13 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:13.144+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:13.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:13 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:13.144+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:40.787093+0000 front 2024-04-23T18:47:40.787132+0000 (oldest deadline 2024-04-23T18:48:06.087168+0000) 2024-04-23T18:48:13.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:13 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:13.416+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:13.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:13 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:13.416+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:13.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:13 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:13.416+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:13.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:13 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:13.416+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:13.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:13 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:13.416+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:13.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:13 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:13.416+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:14.440 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:14 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:14.149+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:14.440 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:14 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:14.149+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:14.440 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:14 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:14.149+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:14.440 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:14 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:14.149+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:14.440 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:14 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:14.149+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:14.440 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:14 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:14.149+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:14.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:14 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:14.145+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:14.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:14 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:14.145+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:14.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:14 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:14.145+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:14.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:14 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:14.145+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:14.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:14 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:14.145+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:40.787093+0000 front 2024-04-23T18:47:40.787132+0000 (oldest deadline 2024-04-23T18:48:06.087168+0000) 2024-04-23T18:48:14.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:14 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:14.381+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:14.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:14 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:14.381+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:14.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:14 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:14.381+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:14.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:14 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:14.381+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:14.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:14 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:14.381+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:14.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:14 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:14.381+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:15.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:15 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:15.190+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:15.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:15 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:15.190+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:15.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:15 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:15.190+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:15.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:15 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:15.190+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:15.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:15 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:15.190+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:15.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:15 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:15.190+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:15.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:15 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:15.185+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:15.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:15 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:15.185+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:15.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:15 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:15.185+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:15.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:15 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:15.185+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:15.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:15 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:15.185+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:40.787093+0000 front 2024-04-23T18:47:40.787132+0000 (oldest deadline 2024-04-23T18:48:06.087168+0000) 2024-04-23T18:48:15.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:15 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:15.425+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:15.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:15 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:15.425+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:15.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:15 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:15.425+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:15.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:15 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:15.425+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:15.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:15 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:15.425+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:15.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:15 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:15.425+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:16.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:16 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:16.213+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:16.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:16 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:16.213+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:16.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:16 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:16.213+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:16.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:16 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:16.213+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:16.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:16 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:16.213+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:16.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:16 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:16.213+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:16.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:16 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:16.201+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:02.881322+0000 front 2024-04-23T18:47:02.881342+0000 (oldest deadline 2024-04-23T18:47:24.581524+0000) 2024-04-23T18:48:16.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:16 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:16.201+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:10.383339+0000 front 2024-04-23T18:47:10.383319+0000 (oldest deadline 2024-04-23T18:47:35.083423+0000) 2024-04-23T18:48:16.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:16 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:16.201+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:15.084092+0000 front 2024-04-23T18:47:15.084055+0000 (oldest deadline 2024-04-23T18:47:40.983839+0000) 2024-04-23T18:48:16.610 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:16 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:16.201+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:27.385161+0000 front 2024-04-23T18:47:27.385191+0000 (oldest deadline 2024-04-23T18:47:50.885320+0000) 2024-04-23T18:48:16.611 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:16 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9[63905]: 2024-04-23T18:48:16.201+0000 7f91eab0e700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:40.787093+0000 front 2024-04-23T18:47:40.787132+0000 (oldest deadline 2024-04-23T18:48:06.087168+0000) 2024-04-23T18:48:16.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:16 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:16.437+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:16.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:16 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:16.437+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:16.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:16 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:16.437+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:16.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:16 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:16.437+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:16.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:16 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:16.437+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:16.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:16 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:16.437+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:16.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:16 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:16.437+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.329689+0000 front 2024-04-23T18:47:51.329660+0000 (oldest deadline 2024-04-23T18:48:16.029752+0000) 2024-04-23T18:48:16.946 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:16 smithi132 podman[89989]: 2024-04-23 18:48:16.64830385 +0000 UTC m=+5.471139480 container died 1c9bfb1d7c69327aa178adeb94156a74e6b156e04a8e4abc94a13d736f001b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-23T18:48:16.947 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:16 smithi132 podman[89989]: 2024-04-23 18:48:16.956442382 +0000 UTC m=+5.779278011 container cleanup 1c9bfb1d7c69327aa178adeb94156a74e6b156e04a8e4abc94a13d736f001b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T18:48:17.360 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:16 smithi132 bash[89989]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9 2024-04-23T18:48:17.360 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:17 smithi132 podman[90006]: 2024-04-23 18:48:17.174221673 +0000 UTC m=+0.528252774 container remove 1c9bfb1d7c69327aa178adeb94156a74e6b156e04a8e4abc94a13d736f001b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-23T18:48:17.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:17 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:17.205+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:17.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:17 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:17.205+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:17.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:17 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:17.205+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:17.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:17 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:17.205+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:17.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:17 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:17.205+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:17.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:17 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:17.205+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:17.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:17 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:17.426+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:17.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:17 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:17.426+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:17.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:17 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:17.426+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:17.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:17 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:17.426+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:17.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:17 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:17.426+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:17.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:17 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:17.426+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:17.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:17 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:17.426+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.329689+0000 front 2024-04-23T18:47:51.329660+0000 (oldest deadline 2024-04-23T18:48:16.029752+0000) 2024-04-23T18:48:17.776 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:17 smithi132 podman[90101]: 2024-04-23 18:48:17.400343807 +0000 UTC m=+0.018293161 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:48:17.776 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:17 smithi132 podman[90101]: 2024-04-23 18:48:17.499940839 +0000 UTC m=+0.117890186 container create 766e57c1b3479839c383a17fd6cd0b48f98640c7942db8dcc7e629ad0cd19b13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:48:17.776 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:17 smithi132 podman[90101]: 2024-04-23 18:48:17.72762585 +0000 UTC m=+0.345575203 container init 766e57c1b3479839c383a17fd6cd0b48f98640c7942db8dcc7e629ad0cd19b13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-23T18:48:17.776 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:17 smithi132 podman[90101]: 2024-04-23 18:48:17.730873957 +0000 UTC m=+0.348823303 container start 766e57c1b3479839c383a17fd6cd0b48f98640c7942db8dcc7e629ad0cd19b13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-23T18:48:17.776 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:17 smithi132 podman[90101]: 2024-04-23 18:48:17.785605495 +0000 UTC m=+0.403554859 container attach 766e57c1b3479839c383a17fd6cd0b48f98640c7942db8dcc7e629ad0cd19b13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-23T18:48:18.030 INFO:journalctl@ceph.osd.9.smithi132.stdout:Apr 23 18:48:18 smithi132 podman[90101]: 2024-04-23 18:48:18.018214965 +0000 UTC m=+0.636164311 container died 766e57c1b3479839c383a17fd6cd0b48f98640c7942db8dcc7e629ad0cd19b13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:48:18.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:18 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:18.223+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:18.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:18 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:18.223+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:18.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:18 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:18.223+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:18.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:18 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:18.223+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:18.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:18 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:18.223+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:18.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:18 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:18.223+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:18.581 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.9.service' 2024-04-23T18:48:18.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:48:18.620 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-23T18:48:18.620 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-23T18:48:18.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.10 2024-04-23T18:48:18.646 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:18 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:18.400+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:18.646 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:18 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:18.400+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:18.646 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:18 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:18.400+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:18.646 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:18 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:18.400+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:18.647 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:18 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:18.400+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:18.647 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:18 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:18.400+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:18.647 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:18 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:18.400+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.329689+0000 front 2024-04-23T18:47:51.329660+0000 (oldest deadline 2024-04-23T18:48:16.029752+0000) 2024-04-23T18:48:18.929 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:18 smithi148 systemd[1]: Stopping Ceph osd.10 for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:48:19.263 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:18 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:18.936+0000 7fae28fee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:48:19.263 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:18 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:18.936+0000 7fae28fee700 -1 osd.10 102 *** Got signal Terminated *** 2024-04-23T18:48:19.263 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:18 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:18.936+0000 7fae28fee700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:48:19.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:19 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:19.196+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:19.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:19 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:19.196+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:19.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:19 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:19.196+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:19.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:19 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:19.196+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:19.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:19 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:19.196+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:19.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:19 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:19.196+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:19.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:19 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:19.385+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:19.910 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:19 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:19.385+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:19.910 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:19 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:19.385+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:19.910 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:19 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:19.385+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:19.910 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:19 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:19.385+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:19.910 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:19 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:19.385+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:19.911 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:19 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:19.385+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.329689+0000 front 2024-04-23T18:47:51.329660+0000 (oldest deadline 2024-04-23T18:48:16.029752+0000) 2024-04-23T18:48:20.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:20 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:20.172+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:20.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:20 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:20.172+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:20.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:20 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:20.172+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:20.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:20 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:20.172+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:20.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:20 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:20.172+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:20.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:20 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:20.172+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:20.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:20 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:20.367+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:20.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:20 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:20.367+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:20.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:20 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:20.367+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:20.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:20 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:20.367+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:20.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:20 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:20.367+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:20.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:20 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:20.367+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:20.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:20 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:20.367+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.329689+0000 front 2024-04-23T18:47:51.329660+0000 (oldest deadline 2024-04-23T18:48:16.029752+0000) 2024-04-23T18:48:21.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:21 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:21.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:21.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:21 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:21.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:21.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:21 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:21.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:21.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:21 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:21.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:21.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:21 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:21.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:21.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:21 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:21.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:21.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:21 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:21.384+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:21.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:21 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:21.384+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:21.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:21 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:21.384+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:21.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:21 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:21.384+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:21.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:21 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:21.384+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:21.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:21 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:21.384+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:21.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:21 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:21.384+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.329689+0000 front 2024-04-23T18:47:51.329660+0000 (oldest deadline 2024-04-23T18:48:16.029752+0000) 2024-04-23T18:48:22.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:22 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:22.148+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:22.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:22 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:22.148+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:22.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:22 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:22.148+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:22.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:22 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:22.148+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:22.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:22 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:22.148+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:22.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:22 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:22.148+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:22.718 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:22 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:22.356+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:22.719 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:22 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:22.356+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:22.719 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:22 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:22.356+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:22.719 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:22 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:22.356+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:22.719 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:22 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:22.356+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:22.719 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:22 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:22.356+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:22.719 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:22 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:22.356+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.329689+0000 front 2024-04-23T18:47:51.329660+0000 (oldest deadline 2024-04-23T18:48:16.029752+0000) 2024-04-23T18:48:23.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:23 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:23.194+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:23.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:23 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:23.194+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:23.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:23 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:23.194+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:23.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:23 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:23.194+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:23.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:23 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:23.194+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:23.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:23 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:23.194+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:23.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:23 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:23.194+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-23T18:47:56.421881+0000 front 2024-04-23T18:47:56.421918+0000 (oldest deadline 2024-04-23T18:48:22.321809+0000) 2024-04-23T18:48:23.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:23 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:23.359+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:47:00.121190+0000 front 2024-04-23T18:47:00.121144+0000 (oldest deadline 2024-04-23T18:47:24.221098+0000) 2024-04-23T18:48:23.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:23 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:23.359+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.623949+0000 front 2024-04-23T18:47:11.624015+0000 (oldest deadline 2024-04-23T18:47:37.524145+0000) 2024-04-23T18:48:23.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:23 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:23.359+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:18.625399+0000 front 2024-04-23T18:47:18.625454+0000 (oldest deadline 2024-04-23T18:47:40.325303+0000) 2024-04-23T18:48:23.763 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:23 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:23.359+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-04-23T18:47:23.826287+0000 front 2024-04-23T18:47:23.826374+0000 (oldest deadline 2024-04-23T18:47:49.126200+0000) 2024-04-23T18:48:23.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:23 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:23.359+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:34.927699+0000 front 2024-04-23T18:47:34.927631+0000 (oldest deadline 2024-04-23T18:48:00.827787+0000) 2024-04-23T18:48:23.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:23 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:23.359+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:43.128895+0000 front 2024-04-23T18:47:43.128982+0000 (oldest deadline 2024-04-23T18:48:07.228764+0000) 2024-04-23T18:48:23.764 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:23 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10[57302]: 2024-04-23T18:48:23.359+0000 7fae255e2700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.329689+0000 front 2024-04-23T18:47:51.329660+0000 (oldest deadline 2024-04-23T18:48:16.029752+0000) 2024-04-23T18:48:24.504 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:24 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:24.242+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:24.504 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:24 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:24.242+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:24.504 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:24 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:24.242+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:24.504 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:24 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:24.242+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:24.504 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:24 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:24.242+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:24.504 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:24 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:24.242+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:24.505 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:24 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:24.242+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-23T18:47:56.421881+0000 front 2024-04-23T18:47:56.421918+0000 (oldest deadline 2024-04-23T18:48:22.321809+0000) 2024-04-23T18:48:24.703 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:24 smithi148 podman[68868]: 2024-04-23 18:48:24.416849285 +0000 UTC m=+5.590993028 container died b767055a7ec45739486050dea5830d1534d4aa00ca59ee6964523fdfb27c5629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-23T18:48:24.978 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:24 smithi148 podman[68868]: 2024-04-23 18:48:24.795792922 +0000 UTC m=+5.969936665 container cleanup b767055a7ec45739486050dea5830d1534d4aa00ca59ee6964523fdfb27c5629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-23T18:48:24.979 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:24 smithi148 bash[68868]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10 2024-04-23T18:48:25.263 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:24 smithi148 podman[68886]: 2024-04-23 18:48:24.98851333 +0000 UTC m=+0.560112566 container remove b767055a7ec45739486050dea5830d1534d4aa00ca59ee6964523fdfb27c5629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:48:25.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:25 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:25.280+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:25.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:25 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:25.280+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:25.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:25 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:25.280+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:25.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:25 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:25.280+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:25.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:25 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:25.280+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:25.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:25 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:25.280+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:25.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:25 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:25.280+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-23T18:47:56.421881+0000 front 2024-04-23T18:47:56.421918+0000 (oldest deadline 2024-04-23T18:48:22.321809+0000) 2024-04-23T18:48:25.723 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:25 smithi148 podman[68972]: 2024-04-23 18:48:25.314844363 +0000 UTC m=+0.018062081 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:48:25.723 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:25 smithi148 podman[68972]: 2024-04-23 18:48:25.42246009 +0000 UTC m=+0.125677808 container create 6c456491e7b520d436c49f32dd8c462fc1474d8b6d59cf09db030e93088751b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-23T18:48:25.723 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:25 smithi148 podman[68972]: 2024-04-23 18:48:25.64975357 +0000 UTC m=+0.352971288 container init 6c456491e7b520d436c49f32dd8c462fc1474d8b6d59cf09db030e93088751b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-23T18:48:25.723 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:25 smithi148 podman[68972]: 2024-04-23 18:48:25.653378744 +0000 UTC m=+0.356596465 container start 6c456491e7b520d436c49f32dd8c462fc1474d8b6d59cf09db030e93088751b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:48:25.723 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:25 smithi148 podman[68972]: 2024-04-23 18:48:25.732970715 +0000 UTC m=+0.436188449 container attach 6c456491e7b520d436c49f32dd8c462fc1474d8b6d59cf09db030e93088751b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-23T18:48:26.013 INFO:journalctl@ceph.osd.10.smithi148.stdout:Apr 23 18:48:25 smithi148 podman[68972]: 2024-04-23 18:48:25.950784443 +0000 UTC m=+0.654002162 container died 6c456491e7b520d436c49f32dd8c462fc1474d8b6d59cf09db030e93088751b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-23T18:48:26.522 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.10.service' 2024-04-23T18:48:26.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:26 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:26.267+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:26.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:26 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:26.267+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:26.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:26 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:26.267+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:26.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:26 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:26.267+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:26.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:26 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:26.267+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:26.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:26 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:26.267+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:26.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:26 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:26.267+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-23T18:47:56.421881+0000 front 2024-04-23T18:47:56.421918+0000 (oldest deadline 2024-04-23T18:48:22.321809+0000) 2024-04-23T18:48:26.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:48:26.601 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-23T18:48:26.602 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-23T18:48:26.602 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.11 2024-04-23T18:48:26.907 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:26 smithi157 systemd[1]: Stopping Ceph osd.11 for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:48:27.250 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:26 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:26.916+0000 7f0aaf568700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:48:27.250 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:26 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:26.916+0000 7f0aaf568700 -1 osd.11 102 *** Got signal Terminated *** 2024-04-23T18:48:27.250 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:26 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:26.916+0000 7f0aaf568700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:48:27.250 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:27 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:27.259+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:27.250 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:27 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:27.259+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:27.250 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:27 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:27.259+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:27.250 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:27 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:27.259+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:27.250 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:27 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:27.259+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:27.251 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:27 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:27.259+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:27.251 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:27 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:27.259+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-23T18:47:56.421881+0000 front 2024-04-23T18:47:56.421918+0000 (oldest deadline 2024-04-23T18:48:22.321809+0000) 2024-04-23T18:48:28.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:28 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:28.225+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:28.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:28 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:28.225+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:28.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:28 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:28.225+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:28.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:28 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:28.225+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:28.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:28 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:28.225+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:28.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:28 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:28.225+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:28.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:28 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:28.225+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-23T18:47:56.421881+0000 front 2024-04-23T18:47:56.421918+0000 (oldest deadline 2024-04-23T18:48:22.321809+0000) 2024-04-23T18:48:29.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:29 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:29.184+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:29.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:29 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:29.184+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:29.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:29 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:29.184+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:29.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:29 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:29.184+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:29.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:29 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:29.184+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:29.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:29 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:29.184+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:29.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:29 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:29.184+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-23T18:47:56.421881+0000 front 2024-04-23T18:47:56.421918+0000 (oldest deadline 2024-04-23T18:48:22.321809+0000) 2024-04-23T18:48:30.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:30 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:30.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:30.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:30 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:30.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:30.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:30 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:30.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:30.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:30 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:30.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:30.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:30 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:30.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:30.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:30 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:30.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:30.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:30 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:30.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-23T18:47:56.421881+0000 front 2024-04-23T18:47:56.421918+0000 (oldest deadline 2024-04-23T18:48:22.321809+0000) 2024-04-23T18:48:30.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:30 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:30.140+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-04-23T18:48:05.122990+0000 front 2024-04-23T18:48:05.123029+0000 (oldest deadline 2024-04-23T18:48:29.723582+0000) 2024-04-23T18:48:31.538 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:31 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:31.158+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-04-23T18:46:59.111443+0000 front 2024-04-23T18:46:59.111508+0000 (oldest deadline 2024-04-23T18:47:24.912289+0000) 2024-04-23T18:48:31.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:31 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:31.158+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-04-23T18:47:11.214945+0000 front 2024-04-23T18:47:11.214860+0000 (oldest deadline 2024-04-23T18:47:36.514613+0000) 2024-04-23T18:48:31.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:31 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:31.158+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-04-23T18:47:17.015569+0000 front 2024-04-23T18:47:17.015431+0000 (oldest deadline 2024-04-23T18:47:42.915414+0000) 2024-04-23T18:48:31.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:31 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:31.158+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-04-23T18:47:35.717851+0000 front 2024-04-23T18:47:35.717944+0000 (oldest deadline 2024-04-23T18:47:59.217767+0000) 2024-04-23T18:48:31.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:31 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:31.158+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-04-23T18:47:41.518563+0000 front 2024-04-23T18:47:41.518662+0000 (oldest deadline 2024-04-23T18:48:04.919202+0000) 2024-04-23T18:48:31.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:31 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:31.158+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-04-23T18:47:51.920807+0000 front 2024-04-23T18:47:51.920798+0000 (oldest deadline 2024-04-23T18:48:13.020486+0000) 2024-04-23T18:48:31.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:31 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:31.158+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-04-23T18:47:56.421881+0000 front 2024-04-23T18:47:56.421918+0000 (oldest deadline 2024-04-23T18:48:22.321809+0000) 2024-04-23T18:48:31.539 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:31 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11[57825]: 2024-04-23T18:48:31.158+0000 7f0aabb5c700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-04-23T18:48:05.122990+0000 front 2024-04-23T18:48:05.123029+0000 (oldest deadline 2024-04-23T18:48:29.723582+0000) 2024-04-23T18:48:32.657 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:32 smithi157 podman[69895]: 2024-04-23 18:48:32.356590849 +0000 UTC m=+5.555315699 container died fc9b31f7135946a4ede6d2390e7ea5f85e529110f20f53e71d00b90ce5dd25b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-23T18:48:33.015 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:32 smithi157 podman[69895]: 2024-04-23 18:48:32.749174079 +0000 UTC m=+5.947898927 container cleanup fc9b31f7135946a4ede6d2390e7ea5f85e529110f20f53e71d00b90ce5dd25b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-23T18:48:33.015 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:32 smithi157 bash[69895]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11 2024-04-23T18:48:33.288 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:33 smithi157 podman[69913]: 2024-04-23 18:48:33.025123963 +0000 UTC m=+0.663927635 container remove fc9b31f7135946a4ede6d2390e7ea5f85e529110f20f53e71d00b90ce5dd25b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-23T18:48:33.628 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:33 smithi157 podman[70003]: 2024-04-23 18:48:33.271764853 +0000 UTC m=+0.018341807 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:48:33.628 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:33 smithi157 podman[70003]: 2024-04-23 18:48:33.392703512 +0000 UTC m=+0.139280502 container create a5944bd0ba14edd1de9fd0dc97845f078be2cac50386ff0a4a82b9d98f905475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:48:33.628 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:33 smithi157 podman[70003]: 2024-04-23 18:48:33.562573293 +0000 UTC m=+0.309150243 container init a5944bd0ba14edd1de9fd0dc97845f078be2cac50386ff0a4a82b9d98f905475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:48:33.628 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:33 smithi157 podman[70003]: 2024-04-23 18:48:33.565854633 +0000 UTC m=+0.312431581 container start a5944bd0ba14edd1de9fd0dc97845f078be2cac50386ff0a4a82b9d98f905475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-23T18:48:33.628 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:33 smithi157 podman[70003]: 2024-04-23 18:48:33.636983847 +0000 UTC m=+0.383560807 container attach a5944bd0ba14edd1de9fd0dc97845f078be2cac50386ff0a4a82b9d98f905475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-23T18:48:34.038 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:33 smithi157 podman[70003]: 2024-04-23 18:48:33.857008208 +0000 UTC m=+0.603585156 container died a5944bd0ba14edd1de9fd0dc97845f078be2cac50386ff0a4a82b9d98f905475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-23T18:48:34.407 INFO:journalctl@ceph.osd.11.smithi157.stdout:Apr 23 18:48:34 smithi157 podman[70029]: 2024-04-23 18:48:34.415298914 +0000 UTC m=+0.552976266 container remove a5944bd0ba14edd1de9fd0dc97845f078be2cac50386ff0a4a82b9d98f905475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:48:34.429 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@osd.11.service' 2024-04-23T18:48:34.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:48:34.465 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-23T18:48:34.466 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-23T18:48:34.466 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.a 2024-04-23T18:48:34.765 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:48:34 smithi132 systemd[1]: Stopping Ceph mds.a for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:48:35.110 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:48:34 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-a[70735]: 2024-04-23T18:48:34.774+0000 7fdca03e1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:48:35.110 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:48:34 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-a[70735]: 2024-04-23T18:48:34.774+0000 7fdca03e1700 -1 mds.a *** got signal Terminated *** 2024-04-23T18:48:39.098 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:48:38 smithi132 podman[90222]: 2024-04-23 18:48:38.779068495 +0000 UTC m=+4.104675977 container died d826cab44e48fdb52bccce1c7e17724eb9c10c606cea554310e7a62c76b7b4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-04-23T18:48:39.360 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:48:39 smithi132 podman[90222]: 2024-04-23 18:48:39.207281832 +0000 UTC m=+4.532889312 container cleanup d826cab44e48fdb52bccce1c7e17724eb9c10c606cea554310e7a62c76b7b4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-23T18:48:39.360 INFO:journalctl@ceph.mds.a.smithi132.stdout:Apr 23 18:48:39 smithi132 bash[90222]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-a 2024-04-23T18:48:39.615 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.a.service' 2024-04-23T18:48:39.647 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:48:39.648 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-23T18:48:39.648 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-23T18:48:39.648 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.d 2024-04-23T18:48:39.993 INFO:journalctl@ceph.mds.d.smithi132.stdout:Apr 23 18:48:39 smithi132 systemd[1]: Stopping Ceph mds.d for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:48:40.360 INFO:journalctl@ceph.mds.d.smithi132.stdout:Apr 23 18:48:40 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-d[69782]: 2024-04-23T18:48:40.002+0000 7f84cf172700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:48:40.361 INFO:journalctl@ceph.mds.d.smithi132.stdout:Apr 23 18:48:40 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-d[69782]: 2024-04-23T18:48:40.002+0000 7f84cf172700 -1 mds.d *** got signal Terminated *** 2024-04-23T18:48:43.993 INFO:journalctl@ceph.mds.d.smithi132.stdout:Apr 23 18:48:43 smithi132 podman[90343]: 2024-04-23 18:48:43.658177318 +0000 UTC m=+3.786476218 container died a61f079d5fe74d8f0914ec72e08b8e1f0eef5ef6befd31305277fa781965f967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-23T18:48:44.302 INFO:journalctl@ceph.mds.d.smithi132.stdout:Apr 23 18:48:44 smithi132 podman[90343]: 2024-04-23 18:48:44.094251072 +0000 UTC m=+4.222549978 container cleanup a61f079d5fe74d8f0914ec72e08b8e1f0eef5ef6befd31305277fa781965f967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:48:44.302 INFO:journalctl@ceph.mds.d.smithi132.stdout:Apr 23 18:48:44 smithi132 bash[90343]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-d 2024-04-23T18:48:44.553 INFO:journalctl@ceph.mds.d.smithi132.stdout:Apr 23 18:48:44 smithi132 podman[90357]: 2024-04-23 18:48:44.312035336 +0000 UTC m=+0.652810417 container remove a61f079d5fe74d8f0914ec72e08b8e1f0eef5ef6befd31305277fa781965f967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:48:44.556 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.d.service' 2024-04-23T18:48:44.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:48:44.589 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-23T18:48:44.589 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-23T18:48:44.589 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.g 2024-04-23T18:48:44.860 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:48:44 smithi132 systemd[1]: Stopping Ceph mds.g for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:48:45.360 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:48:44 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-g[70379]: 2024-04-23T18:48:44.955+0000 7fa5ae49b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:48:45.360 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:48:44 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-g[70379]: 2024-04-23T18:48:44.956+0000 7fa5ae49b700 -1 mds.g *** got signal Terminated *** 2024-04-23T18:48:47.455 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:48:47 smithi132 podman[90465]: 2024-04-23 18:48:47.169125962 +0000 UTC m=+2.346675673 container died fcb9452407eab67c9a29a376caef143514176a2d934d830de7e0075c0da10ab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:48:47.789 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:48:47 smithi132 podman[90465]: 2024-04-23 18:48:47.531527255 +0000 UTC m=+2.709076973 container cleanup fcb9452407eab67c9a29a376caef143514176a2d934d830de7e0075c0da10ab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T18:48:47.789 INFO:journalctl@ceph.mds.g.smithi132.stdout:Apr 23 18:48:47 smithi132 bash[90465]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-g 2024-04-23T18:48:47.942 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.g.service' 2024-04-23T18:48:47.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:48:47.973 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-23T18:48:47.973 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-23T18:48:47.973 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.j 2024-04-23T18:48:48.360 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:48:48 smithi132 systemd[1]: Stopping Ceph mds.j for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:48:48.360 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:48:48 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-j[69395]: 2024-04-23T18:48:48.292+0000 7fcca3f13700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:48:48.360 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:48:48 smithi132 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-j[69395]: 2024-04-23T18:48:48.292+0000 7fcca3f13700 -1 mds.j *** got signal Terminated *** 2024-04-23T18:48:52.610 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:48:52 smithi132 podman[90585]: 2024-04-23 18:48:52.29875917 +0000 UTC m=+4.100098838 container died 59874b8d89d27ac8fa3af79c0b45ea12c54619e5e595dffd1a4457be4c5b92b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-04-23T18:48:52.968 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:48:52 smithi132 podman[90585]: 2024-04-23 18:48:52.735027708 +0000 UTC m=+4.536367371 container cleanup 59874b8d89d27ac8fa3af79c0b45ea12c54619e5e595dffd1a4457be4c5b92b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-23T18:48:52.968 INFO:journalctl@ceph.mds.j.smithi132.stdout:Apr 23 18:48:52 smithi132 bash[90585]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-j 2024-04-23T18:48:53.193 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.j.service' 2024-04-23T18:48:53.223 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:48:53.223 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-23T18:48:53.223 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-23T18:48:53.223 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.b 2024-04-23T18:48:53.513 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:48:53 smithi148 systemd[1]: Stopping Ceph mds.b for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:48:53.776 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:48:53 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b[62525]: 2024-04-23T18:48:53.711+0000 7fa670302700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:48:53.777 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:48:53 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b[62525]: 2024-04-23T18:48:53.711+0000 7fa670302700 -1 mds.b *** got signal Terminated *** 2024-04-23T18:48:55.513 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:48:55 smithi148 podman[69077]: 2024-04-23 18:48:55.169024182 +0000 UTC m=+1.744842298 container died 9219f4346f4cc423ed2e410393351956263eecf7f2f685bc37f708328077775c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:48:56.013 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:48:55 smithi148 podman[69077]: 2024-04-23 18:48:55.835604166 +0000 UTC m=+2.411422283 container cleanup 9219f4346f4cc423ed2e410393351956263eecf7f2f685bc37f708328077775c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-23T18:48:56.013 INFO:journalctl@ceph.mds.b.smithi148.stdout:Apr 23 18:48:55 smithi148 bash[69077]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-b 2024-04-23T18:48:56.249 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.b.service' 2024-04-23T18:48:56.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:48:56.284 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-23T18:48:56.285 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-23T18:48:56.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.e 2024-04-23T18:48:56.513 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:48:56 smithi148 systemd[1]: Stopping Ceph mds.e for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:48:57.013 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:48:56 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-e[61906]: 2024-04-23T18:48:56.623+0000 7fef71f2a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:48:57.013 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:48:56 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-e[61906]: 2024-04-23T18:48:56.623+0000 7fef71f2a700 -1 mds.e *** got signal Terminated *** 2024-04-23T18:48:59.430 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:48:59 smithi148 podman[69208]: 2024-04-23 18:48:59.173384829 +0000 UTC m=+2.665295419 container died 2c46f700e04ffea43a0eb21adc45d03dbf5636003d4db9492cf1310baf7b4055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:48:59.763 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:48:59 smithi148 podman[69208]: 2024-04-23 18:48:59.539346485 +0000 UTC m=+3.031257080 container cleanup 2c46f700e04ffea43a0eb21adc45d03dbf5636003d4db9492cf1310baf7b4055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T18:48:59.763 INFO:journalctl@ceph.mds.e.smithi148.stdout:Apr 23 18:48:59 smithi148 bash[69208]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-e 2024-04-23T18:49:00.007 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.e.service' 2024-04-23T18:49:00.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:49:00.039 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-23T18:49:00.039 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-23T18:49:00.040 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.h 2024-04-23T18:49:00.367 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:49:00 smithi148 systemd[1]: Stopping Ceph mds.h for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:49:00.763 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:49:00 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-h[62217]: 2024-04-23T18:49:00.375+0000 7f2e77b2d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:49:00.764 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:49:00 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-h[62217]: 2024-04-23T18:49:00.375+0000 7f2e77b2d700 -1 mds.h *** got signal Terminated *** 2024-04-23T18:49:03.426 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:49:03 smithi148 podman[69328]: 2024-04-23 18:49:03.18248892 +0000 UTC m=+2.909236966 container died a50d964884fa0352d4e1cf3cdcc077390deeb76721879fbee02c64d5d713c783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:49:03.763 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:49:03 smithi148 podman[69328]: 2024-04-23 18:49:03.534674901 +0000 UTC m=+3.261422946 container cleanup a50d964884fa0352d4e1cf3cdcc077390deeb76721879fbee02c64d5d713c783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-23T18:49:03.763 INFO:journalctl@ceph.mds.h.smithi148.stdout:Apr 23 18:49:03 smithi148 bash[69328]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-h 2024-04-23T18:49:03.910 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.h.service' 2024-04-23T18:49:03.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:49:03.941 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-23T18:49:03.941 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-23T18:49:03.941 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.k 2024-04-23T18:49:04.204 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:49:04 smithi148 systemd[1]: Stopping Ceph mds.k for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:49:04.513 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:49:04 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-k[61588]: 2024-04-23T18:49:04.263+0000 7f966c2a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:49:04.513 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:49:04 smithi148 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-k[61588]: 2024-04-23T18:49:04.263+0000 7f966c2a1700 -1 mds.k *** got signal Terminated *** 2024-04-23T18:49:07.455 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:49:07 smithi148 podman[69449]: 2024-04-23 18:49:07.18252862 +0000 UTC m=+3.017084691 container died 1c7c47ae3cda81e321a523b47d12ffba6ca810338ca8d49f99a84bde86e65544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-23T18:49:07.722 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:49:07 smithi148 podman[69449]: 2024-04-23 18:49:07.564609881 +0000 UTC m=+3.399165945 container cleanup 1c7c47ae3cda81e321a523b47d12ffba6ca810338ca8d49f99a84bde86e65544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-23T18:49:07.722 INFO:journalctl@ceph.mds.k.smithi148.stdout:Apr 23 18:49:07 smithi148 bash[69449]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-k 2024-04-23T18:49:07.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.k.service' 2024-04-23T18:49:07.964 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:49:07.964 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-23T18:49:07.964 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-23T18:49:07.964 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.c 2024-04-23T18:49:08.288 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:49:08 smithi157 systemd[1]: Stopping Ceph mds.c for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:49:08.788 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:49:08 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-c[63192]: 2024-04-23T18:49:08.381+0000 7f7accc0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:49:08.789 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:49:08 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-c[63192]: 2024-04-23T18:49:08.381+0000 7f7accc0d700 -1 mds.c *** got signal Terminated *** 2024-04-23T18:49:11.439 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:49:11 smithi157 podman[70108]: 2024-04-23 18:49:11.167234721 +0000 UTC m=+3.003046707 container died e805f720b6eed34274a2f74f5d4a695d384a87a072e39c21bdfe302bc010a913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-23T18:49:11.788 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:49:11 smithi157 podman[70108]: 2024-04-23 18:49:11.556265805 +0000 UTC m=+3.392077783 container cleanup e805f720b6eed34274a2f74f5d4a695d384a87a072e39c21bdfe302bc010a913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:49:11.788 INFO:journalctl@ceph.mds.c.smithi157.stdout:Apr 23 18:49:11 smithi157 bash[70108]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-c 2024-04-23T18:49:11.969 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.c.service' 2024-04-23T18:49:12.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:49:12.004 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-23T18:49:12.004 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-23T18:49:12.004 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.f 2024-04-23T18:49:12.326 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:49:12 smithi157 systemd[1]: Stopping Ceph mds.f for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:49:12.788 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:49:12 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f[62879]: 2024-04-23T18:49:12.335+0000 7f722b5c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:49:12.789 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:49:12 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f[62879]: 2024-04-23T18:49:12.335+0000 7f722b5c9700 -1 mds.f *** got signal Terminated *** 2024-04-23T18:49:15.460 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:49:15 smithi157 podman[70242]: 2024-04-23 18:49:15.181530336 +0000 UTC m=+2.950263677 container died fdfc1d81fa7fa7291df9f8c7526ad029fbbc9116728a1ce3792e56e3add5a545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-04-23T18:49:15.785 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:49:15 smithi157 podman[70242]: 2024-04-23 18:49:15.585138327 +0000 UTC m=+3.353871656 container cleanup fdfc1d81fa7fa7291df9f8c7526ad029fbbc9116728a1ce3792e56e3add5a545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-23T18:49:15.785 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:49:15 smithi157 bash[70242]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f 2024-04-23T18:49:16.009 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.f.service' 2024-04-23T18:49:16.038 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:49:15 smithi157 podman[70256]: 2024-04-23 18:49:15.794720641 +0000 UTC m=+0.612888029 container remove fdfc1d81fa7fa7291df9f8c7526ad029fbbc9116728a1ce3792e56e3add5a545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-23T18:49:16.038 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:49:16 smithi157 systemd[1]: ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.f.service: Deactivated successfully. 2024-04-23T18:49:16.039 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:49:16 smithi157 systemd[1]: Stopped Ceph mds.f for c34642b4-019e-11ef-bc93-c7b262605968. 2024-04-23T18:49:16.039 INFO:journalctl@ceph.mds.f.smithi157.stdout:Apr 23 18:49:16 smithi157 systemd[1]: ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.f.service: Consumed 3.092s CPU time. 2024-04-23T18:49:16.041 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:49:16.041 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-23T18:49:16.041 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-23T18:49:16.042 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.i 2024-04-23T18:49:16.362 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:49:16 smithi157 systemd[1]: Stopping Ceph mds.i for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:49:16.788 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:49:16 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-i[63503]: 2024-04-23T18:49:16.372+0000 7fbc0b9c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:49:16.788 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:49:16 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-i[63503]: 2024-04-23T18:49:16.372+0000 7fbc0b9c4700 -1 mds.i *** got signal Terminated *** 2024-04-23T18:49:19.446 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:49:19 smithi157 podman[70362]: 2024-04-23 18:49:19.168859582 +0000 UTC m=+2.899839375 container died e3360bcde07088809f315a626ffd9067693ed4e99ba5af3840f2246d90355617 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:49:19.788 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:49:19 smithi157 podman[70362]: 2024-04-23 18:49:19.555624988 +0000 UTC m=+3.286604788 container cleanup e3360bcde07088809f315a626ffd9067693ed4e99ba5af3840f2246d90355617 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-04-23T18:49:19.789 INFO:journalctl@ceph.mds.i.smithi157.stdout:Apr 23 18:49:19 smithi157 bash[70362]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-i 2024-04-23T18:49:19.936 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.i.service' 2024-04-23T18:49:19.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:49:19.969 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-23T18:49:19.969 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-23T18:49:19.969 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.l 2024-04-23T18:49:20.274 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:49:20 smithi157 systemd[1]: Stopping Ceph mds.l for c34642b4-019e-11ef-bc93-c7b262605968... 2024-04-23T18:49:20.538 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:49:20 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l[62558]: 2024-04-23T18:49:20.283+0000 7ffb12396700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:49:20.539 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:49:20 smithi157 ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l[62558]: 2024-04-23T18:49:20.283+0000 7ffb12396700 -1 mds.l *** got signal Terminated *** 2024-04-23T18:49:23.482 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:49:23 smithi157 podman[70481]: 2024-04-23 18:49:23.180359003 +0000 UTC m=+2.981687006 container died d2b89506d0e6d58689ab6bc06f0bdb1b8981fdd049817cd06a220b05419ef853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T18:49:23.482 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:49:23 smithi157 podman[70481]: 2024-04-23 18:49:23.491949424 +0000 UTC m=+3.293277433 container cleanup d2b89506d0e6d58689ab6bc06f0bdb1b8981fdd049817cd06a220b05419ef853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:49:23.788 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:49:23 smithi157 bash[70481]: ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l 2024-04-23T18:49:23.788 INFO:journalctl@ceph.mds.l.smithi157.stdout:Apr 23 18:49:23 smithi157 podman[70495]: 2024-04-23 18:49:23.717835929 +0000 UTC m=+0.530791011 container remove d2b89506d0e6d58689ab6bc06f0bdb1b8981fdd049817cd06a220b05419ef853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c34642b4-019e-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-23T18:49:23.848 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c34642b4-019e-11ef-bc93-c7b262605968@mds.l.service' 2024-04-23T18:49:23.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:49:23.879 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-23T18:49:23.880 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c34642b4-019e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T18:49:24.069 INFO:teuthology.orchestra.run.smithi132.stdout:Deleting cluster with fsid: c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:49:26.537 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c34642b4-019e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T18:49:26.727 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:49:29.233 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c34642b4-019e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T18:49:29.424 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:49:31.727 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:49:31.759 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:49:31.790 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:49:31.815 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-23T18:49:31.817 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122/remote/smithi132/crash 2024-04-23T18:49:31.818 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/crash -- . 2024-04-23T18:49:31.854 INFO:teuthology.orchestra.run.smithi132.stderr:tar: /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T18:49:31.854 INFO:teuthology.orchestra.run.smithi132.stderr:tar: Error is not recoverable: exiting now 2024-04-23T18:49:31.857 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122/remote/smithi148/crash 2024-04-23T18:49:31.858 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/crash -- . 2024-04-23T18:49:31.980 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T18:49:31.980 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now 2024-04-23T18:49:31.982 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122/remote/smithi157/crash 2024-04-23T18:49:31.991 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/crash -- . 2024-04-23T18:49:32.026 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/c34642b4-019e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T18:49:32.027 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2024-04-23T18:49:32.028 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-23T18:49:32.028 DEBUG:teuthology.orchestra.run.smithi132:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-23T18:49:32.067 INFO:tasks.cephadm:Compressing logs... 2024-04-23T18:49:32.067 DEBUG:teuthology.orchestra.run.smithi132:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T18:49:32.111 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T18:49:32.116 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi132.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.log 2024-04-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi132.stderr:: No such file or directory 2024-04-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-23T18:49:32.137 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-23T18:49:32.137 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-23T18:49:32.139 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-23T18:49:32.139 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.4% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi132.stderr: 94.7% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-23T18:49:32.148 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-04-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi148.stderr: -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.log 2024-04-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.5% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.log 2024-04-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi157.stderr:: No such file or directory 2024-04-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.8% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.11.log: 91.9% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T18:49:32.156 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T18:49:32.157 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-23T18:49:32.158 INFO:teuthology.orchestra.run.smithi148.stderr: 91.9% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-23T18:49:32.158 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T18:49:32.159 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-23T18:49:32.159 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T18:49:32.159 INFO:teuthology.orchestra.run.smithi148.stderr: 84.1% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T18:49:32.159 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-23T18:49:32.161 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-23T18:49:32.162 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-23T18:49:32.163 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-23T18:49:32.167 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-23T18:49:32.168 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T18:49:32.170 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T18:49:32.171 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T18:49:32.174 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T18:49:32.180 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.9.log: 91.1% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T18:49:32.193 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T18:49:32.196 INFO:teuthology.orchestra.run.smithi157.stderr: 91.1% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T18:49:32.197 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 93.4% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-23T18:49:32.198 INFO:teuthology.orchestra.run.smithi148.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T18:49:32.199 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-23T18:49:32.201 INFO:teuthology.orchestra.run.smithi132.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T18:49:32.208 INFO:teuthology.orchestra.run.smithi157.stderr: 94.6% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-23T18:49:32.224 INFO:teuthology.orchestra.run.smithi132.stderr: 94.4% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-23T18:49:32.234 INFO:teuthology.orchestra.run.smithi148.stderr: 93.8% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-23T18:49:32.246 INFO:teuthology.orchestra.run.smithi148.stderr: 93.2% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-23T18:49:32.252 INFO:teuthology.orchestra.run.smithi157.stderr: 93.2% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-23T18:49:32.450 INFO:teuthology.orchestra.run.smithi148.stderr: 92.3% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-23T18:49:32.472 INFO:teuthology.orchestra.run.smithi148.stderr: 94.2% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-23T18:49:32.488 INFO:teuthology.orchestra.run.smithi132.stderr: 90.2% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-23T18:49:32.530 INFO:teuthology.orchestra.run.smithi157.stderr: 95.9% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-23T18:49:32.566 INFO:teuthology.orchestra.run.smithi157.stderr: 92.0% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-23T18:49:32.923 INFO:teuthology.orchestra.run.smithi132.stderr: 90.4% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-23T18:49:33.116 INFO:teuthology.orchestra.run.smithi157.stderr: 92.3% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-23T18:49:33.178 INFO:teuthology.orchestra.run.smithi132.stderr: 91.8% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-23T18:49:33.188 INFO:teuthology.orchestra.run.smithi157.stderr: 92.2% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-23T18:49:33.341 INFO:teuthology.orchestra.run.smithi148.stderr: 91.9% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-23T18:49:33.661 INFO:teuthology.orchestra.run.smithi148.stderr: 91.3% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-23T18:49:33.738 INFO:teuthology.orchestra.run.smithi132.stderr: 92.5% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-23T18:49:33.791 INFO:teuthology.orchestra.run.smithi148.stderr: 92.3% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-23T18:49:33.825 INFO:teuthology.orchestra.run.smithi132.stderr: 92.4% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-23T18:49:33.953 INFO:teuthology.orchestra.run.smithi132.stderr: 91.3% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-23T18:49:33.955 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-04-23T18:49:33.955 INFO:teuthology.orchestra.run.smithi132.stderr:real 0m1.831s 2024-04-23T18:49:33.955 INFO:teuthology.orchestra.run.smithi132.stderr:user 0m7.180s 2024-04-23T18:49:33.955 INFO:teuthology.orchestra.run.smithi132.stderr:sys 0m0.257s 2024-04-23T18:49:33.977 INFO:teuthology.orchestra.run.smithi157.stderr: 92.6% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-23T18:49:34.026 INFO:teuthology.orchestra.run.smithi148.stderr: 92.5% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-23T18:49:34.027 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-23T18:49:34.028 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m1.898s 2024-04-23T18:49:34.028 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m6.818s 2024-04-23T18:49:34.028 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m0.259s 2024-04-23T18:49:35.348 INFO:teuthology.orchestra.run.smithi157.stderr: 93.4% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-23T18:49:35.861 INFO:teuthology.orchestra.run.smithi157.stderr: 95.3% -- replaced with /var/log/ceph/c34642b4-019e-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-23T18:49:35.863 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-04-23T18:49:35.863 INFO:teuthology.orchestra.run.smithi157.stderr:real 0m3.732s 2024-04-23T18:49:35.863 INFO:teuthology.orchestra.run.smithi157.stderr:user 0m11.340s 2024-04-23T18:49:35.863 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m0.370s 2024-04-23T18:49:35.863 INFO:tasks.cephadm:Archiving logs... 2024-04-23T18:49:35.864 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122/remote/smithi132/log 2024-04-23T18:49:35.864 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T18:49:37.048 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122/remote/smithi148/log 2024-04-23T18:49:37.048 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T18:49:38.170 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122/remote/smithi157/log 2024-04-23T18:49:38.171 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T18:49:39.664 INFO:tasks.cephadm:Removing cluster... 2024-04-23T18:49:39.664 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c34642b4-019e-11ef-bc93-c7b262605968 --force 2024-04-23T18:49:39.857 INFO:teuthology.orchestra.run.smithi132.stdout:Deleting cluster with fsid: c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:49:40.173 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c34642b4-019e-11ef-bc93-c7b262605968 --force 2024-04-23T18:49:40.364 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:49:40.668 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c34642b4-019e-11ef-bc93-c7b262605968 --force 2024-04-23T18:49:40.859 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: c34642b4-019e-11ef-bc93-c7b262605968 2024-04-23T18:49:41.120 INFO:tasks.cephadm:Removing cephadm ... 2024-04-23T18:49:41.120 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T18:49:41.138 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T18:49:41.156 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T18:49:41.173 INFO:tasks.cephadm:Teardown complete 2024-04-23T18:49:41.173 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T18:49:41.186 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T18:49:41.186 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T18:49:41.189 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T18:49:41.199 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T18:49:41.314 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T18:49:41.324 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T18:49:41.325 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:49:41.328 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:49:41.330 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:49:41.341 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:49:41.342 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:49:41.344 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:49:41.349 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:49:41.349 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-04-23T18:49:41.349 INFO:teuthology.orchestra.run.smithi132.stdout:^- hv01.front.sepia.ceph.com 2 6 377 14 +500us[ +504us] +/- 573ms 2024-04-23T18:49:41.349 INFO:teuthology.orchestra.run.smithi132.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -300us[ -296us] +/- 56ms 2024-04-23T18:49:41.349 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +364us[ +368us] +/- 65ms 2024-04-23T18:49:41.349 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:49:41.351 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:49:41.351 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-04-23T18:49:41.352 INFO:teuthology.orchestra.run.smithi157.stdout:^- hv01.front.sepia.ceph.com 2 6 377 13 +503us[ +503us] +/- 573ms 2024-04-23T18:49:41.352 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -317us[ -317us] +/- 56ms 2024-04-23T18:49:41.352 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +361us[ +361us] +/- 65ms 2024-04-23T18:49:41.352 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:49:41.352 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:49:41.352 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-04-23T18:49:41.352 INFO:teuthology.orchestra.run.smithi148.stdout:^- hv01.front.sepia.ceph.com 2 6 377 15 +518us[ +518us] +/- 573ms 2024-04-23T18:49:41.352 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -311us[ -312us] +/- 56ms 2024-04-23T18:49:41.353 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +361us[ +361us] +/- 65ms 2024-04-23T18:49:41.353 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:49:41.353 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T18:49:41.379 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T18:49:41.379 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T18:49:41.485 DEBUG:teuthology.orchestra.run.smithi132:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T18:49:41.563 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T18:49:41.630 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T18:49:41.695 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:49:41.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:49:41.725 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:49:41.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:49:41.753 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:49:41.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:49:41.780 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T18:49:41.799 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T18:49:41.835 INFO:teuthology.task.internal:Duration was 1878.504218 seconds 2024-04-23T18:49:41.836 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T18:49:41.889 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T18:49:41.889 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T18:49:41.893 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T18:49:41.896 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T18:49:41.929 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:49:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:49:41.932 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:49:42.400 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T18:49:42.400 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2024-04-23T18:49:42.400 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T18:49:42.461 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-04-23T18:49:42.462 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T18:49:42.490 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-04-23T18:49:42.490 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T18:49:42.515 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T18:49:42.515 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T18:49:42.518 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T18:49:42.532 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T18:49:42.582 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T18:49:42.582 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T18:49:42.584 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T18:49:42.600 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T18:49:42.964 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T18:49:42.976 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T18:49:42.976 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T18:49:43.000 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T18:49:43.024 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T18:49:43.049 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T18:49:43.062 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:49:43.064 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:49:43.067 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:49:43.084 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2024-04-23T18:49:43.087 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-04-23T18:49:43.111 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-04-23T18:49:43.157 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:49:43.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:49:43.171 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:49:43.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:49:43.186 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:49:43.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:49:43.211 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T18:49:43.232 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T18:49:43.233 DEBUG:teuthology.misc:Transferring archived files from smithi132:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122/remote/smithi132 2024-04-23T18:49:43.233 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T18:49:43.275 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122/remote/smithi148 2024-04-23T18:49:43.275 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T18:49:43.309 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670122/remote/smithi157 2024-04-23T18:49:43.310 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T18:49:43.345 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T18:49:43.345 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T18:49:43.348 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T18:49:43.350 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T18:49:43.393 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T18:49:43.405 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T18:49:43.405 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T18:49:43.417 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T18:49:43.417 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T18:49:43.420 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T18:49:43.422 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T18:49:43.431 INFO:teuthology.orchestra.run.smithi132.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 18:49 /home/ubuntu/cephtest 2024-04-23T18:49:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 18:49 /home/ubuntu/cephtest 2024-04-23T18:49:43.448 INFO:teuthology.orchestra.run.smithi157.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 18:49 /home/ubuntu/cephtest 2024-04-23T18:49:43.449 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T18:49:43.462 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T18:49:43.624 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/direct_io}} duration: 1878.5042178630829 flavor: default owner: scheduled_leonidus@teuthology success: true 2024-04-23T18:49:43.624 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T18:49:43.698 INFO:teuthology.run:pass